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Abstract. The paper is centered around a new proof of the infinites- 
imal rigidity of smooth closed surfaces with everywhere positive Gauss 
curvature. We use a reformulation that replaces deformation of an em- 
bedding by deformation of the metric inside the body bounded by the 
surface. The proof is obtained by studying derivatives of the Hilbert- 
Einstein functional with boundary term. 

This approach is in a sense dual to proving the Gauss infinitesimal 
rigidity, that is rigidity with respect to the Gauss curvature parametrized 
by the Gauss map, by studying derivatives of the volume bounded by 
the surface. We recall that Blaschke's classical proof of the infinitesimal 
rigidity is also related to the Gauss infinitesimal rigidity, but in a differ- 
ent way: while Blaschke uses Gauss rigidity of the same surface, we use 
the Gauss rigidity of the polar dual. 

In the spherical and in the hyperbolic-de Sitter space, there is a 
perfect duality between the Hilbert-Einstein functional and the volume, 
as well as between both kinds of rigidity. 

We also indicate directions for future research, including the infini- 
tesimal rigidity of convex cores of hyperbolic 3-manifolds. 
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1. Introduction 

1.1. Infinitesimal rigidity of strictly convex smooth surfaces. Let 

M C be a smooth closed surface. An infinitesimal deformation of M 
is a vector field along M, which can be viewed as a map ^: M — )• M'^. An 
infinitesimal deformation produces a family of embeddings (for small t), 

ipt: M ^R^, x + t^. 

Denote by gt the metric induced on M by (/^j. A deformation ^ is called 
isometric, if ^\^_Qgt = 0. Every surface has trivial isometric infinitesimal 
deformations, which are restrictions to M of Killing vector fields in M^. 
We call a surface with everywhere positive Gauss curvature strictly convex. 
Every strictly convex smooth closed surface M is infinitesi- 
mally rigid, that is every isometric infinitesimal deformation 
of M is trivial. 

This theorem was proved by Liebmann [39] for analytic deformations of 
analytic surfaces and by Blaschke [9l [10] and Weyl [58] for C^-deformations 
of C'^-sur faces. For references to other proofs, see [301 §4.1]. 

In this paper we present still another proof. We also describe the general 
framework, historical perspective, and possible further developments. 

1.2. The approach. The infinitesimal rigidity theorem stated above deals 
with a deformation Lpt of an embedding M C M^. However, it is possible to 
reformulate it in terms of deformations of the metric inside the body P C M'^ 
bounded by M. 

View P as an abstract Riemannian manifold equipped with a fiat met- 
ric G induced from . An infinitesimal deformation of G is a field G of 
symmetric bilinear forms on P. An infinitesimal deformation is called cur- 
vature preserving, if the Riemannian metric Gt = G + tG \s flat in the first 
order of t. If G is curvature preserving and vanishes on vectors tangent to 
the boundary, then it induces an isometric infinitesimal deformation ^ of 
the embedding M C M'^, cf. [161 Proposition 3] for a local statement. An 
infinitesimal deformation G is trivial, if it pulls back the metric G by an 
infinitesimal diffeomorphism that restricts to the identity on the boundary: 
G = Cr,G for some vector field rj on P such that tjIqp = 0. 

A surface M C is infinitesimally rigid every curvature 
preserving deformation G such that G\tm = is trivial. 
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The space Gtriv of trivial infinitesimal deformations is very large. There- 
fore it is convenient to consider only G from a subspace tyo C ^ in the space 
of all infinitesimal deformations such that Qtriv + Go = G and such that the 
intersection Gtm H Go is reasonably small. We take as Go the tangent space 
to the space of warped product metrics of the following form: 

~ ,2 2 f g — dr ^ dr\ 
gr = dp + p ( -2 I . 

Here g is the Riemannian metric on M C M'^ , and r : M — > M4, is a smooth 
function, so that gr is a Riemannian metric on x M = R'^ \ {0}. The 
metric gr is flat, if r measures the distance from € (assumed to lie 
within P). For every r, the restriction of Tjr to the surface p = r{x) is g. 

The problem of isometric infinitesimal deformations can now be reformu- 
lated as describing those variations r of the function r that leave the metric 
gr flat in the first order. Trivial variations r arise from moving the coor- 
dinate origin and hence form a space of dimension 3. Note that the origin 
may be a singular point of the metric gr, so that our space Go is not really 
a subspace of G- 

The curvature of a warped product metric gr can be described by a single 
function sec: — >■ M (the sectional curvature in the planes tangent to M). 
Therefore the infinitesimal rigidity of M is equivalent to 

If sec' = 0, then r is trivial. 



1.3. Infinitesimal rigidity of Einstein manifolds. In [33], Koiso proved 
the infinitesimal rigidity of compact closed Einstein manifolds under certain 
restrictions on curvature. His method can be described as the Bochner tech- 
nique applied to the second derivative of the Hilbert-Einstein functional. 
Our proof of the infinitesimal rigidity of strictly convex closed surfaces emu- 
lates Koiso's argument in a certain sense. Koiso also chooses a subspace Go 
in the space of all infinitesimal deformations of the metric, but our choice of 
Go is very specific and works only for manifolds homeomorphic to the ball. 



I. 4. New proof of the infinitesimal rigidity of strictly convex sur- 
faces. The Hilbert-Einstein functional on the space of Riemannian metrics 
on a 3~dimensional compact manifold P with boundary is defined as 

HE(^) = ^ y seal dvol+2 j H darea, 
p dP 

where seal is the scalar curvature of the metric g, and H is the mean curva- 
ture (half of the trace of the shape operator), cf. [221 l7j. In our situation P 
is a ball, and the metric g = gr is a warped product metric, see Subsection 

II. 2[ This allows to express HE as an integral over the boundary M = dP, 
see Theorem 14. 15[ 
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We prove the following formula for the derivative of HE in the direction r: 

/sec 
r darea, 
cos a 

M 

where a is a certain function on M determined by r and g (compare this 
with the well-known formula for HE' for an arbitrary variation of a metric 
^ on a compact closed manifold). 

It follows that the second derivative of HE in the direction r equals 

HE" = / r( ) darea. 

J \ cos a J 

M 

If the variation r is curvature-preserving, then we have sec' = 0. As the 
initial metric Tjr is flat, we also have sec = 0. Hence HE" vanishes. On the 
other hand, an integration by parts yields 

HE" = / 2/idetS darea, 



M 

where h is the support function of M (i. e. distance from to tangent 
planes) and B is the shape operator. A simple algebraic lemma shows that 
deti? < and that det-B vanishes only if B vanishes. As /i > 0, it follows 
that the integral in the last equation vanishes only if i? = 0. Finally, 5 = 
implies that r is a trivial variation. 



1.5. Discrete and smooth, classical and modern. The proof sketched 
in the previuos subsection looks as a logical development of ideas of Koiso 
and others. However, we came to it on a different way, namely from its 
discrete analog p5T]. On the other hand, Blaschke's elegant and somewhat 
mysterious proof \\^ originated from another proof of another theorem that 
used the second derivative of a certain functional. Therefore our proof can 
also be seen as a logical development of ideas of Blaschke and others. 

It turns out that there are two infinitesimal rigidity theorems, metric (the 
one stated above) and Gauss. Metric infinitesimal rigidity is related to the 
Weyl problem in the same way as Gauss infinitesimal rigidity is related to 
the Minkowski problem. Besides, these rigidity theorems are in two ways 
dual to each other. We now proceed to explaining these statements. 

1.6. Blaschke's proof of the infinitesimal rigidity of strictly con- 
vex surfaces. Here is a sketch of the argument from [10] reproduced in 
Subsection 13.41 in a greater detail. 

Every isometric infinitesimal deformation ^ of a surface M C M.^ has an 
associated rotation vector field 77: M — t- M"^. A deformation is trivial if and 
only if its rotation field is constant. One can show that the differential drj 
maps TM to itself (similar to the differential of the field of unit normals). 
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Performing an integration by parts, Blaschke proves the equation 
(1.1) y 2/idet (ir? darea = 0, 

M 

where h is the support function of M. An ingenious argument shows that 
detdry is non-positive, and vanishes only if drj does. As without loss of 
generality h is positive, equation (jl.ip implies dr] = 0. Hence r/ is constant, 
and the infinitesimal deformation ^ is trivial. 

Blaschke's proof can be demystified a bit by observing that d-q = JB, 
where J: TM — t- TM is the rotation by 90°. This explains the above prop- 
erties of det dr] in a more conceptual way. Besides, this relates Blaschke's 
proof to the proof sketched in Subsection 11.41 However, this relation is not 
quite straightforward, as we will see in Subsection 11.91 

1.7. Gauss infinitesimal rigidity and origin of Blaschke's proof. Let 

M C M'^ be a strictly convex smooth closed surface, and let rj: M — )■ be an 
infinitesimal deformation of M. Assume that the embeddings ipt - M ^ M'^, 
ipt{x) = X + tr]{x) preserve the Gauss map: dipt{TxM) \\ T^M. Clearly, 
this is equivalent to dri{TM) C TM. Denote by : M ^ M the Gauss 
curvature of the embedding i/^^, and by K the derivative of Kt aX t = 0. We 
call an infinitesimal deformation r] isogauss, if drj{TM) C TM and K = 0. 
If rj is constant, that is ipt parallelly translates M, then rj is called a trivial 
isogauss infinitesimal deformation. 

Every strictly convex smooth closed surface M is Gauss in- 
finitesimally rigid, that is every isogauss infinitesimal defor- 
mation of M is trivial. 

This theorem is implicitly contained in Hilbert's treatment [26] of Minkowski's 
mixed volumes theory. 

Blaschke [9] observed that certain equations in Hilbert's work coincide 
with equations of the theory of isometric infinitesimal deformations. Weyl 
|58j concretized this observation as follows: 

The rotation vector field of an isometric infinitesimal defor- 
mation of M is an isogauss infinitesimal deformation of M. 

Thus the infinitesimal rigidity theorem from Subsection 11.11 can be proved 
by applying the Minkowski-Hilbert argument to the rotation vector field of 
the deformation. Such a proof was written down by Blaschke in [10] in a 
concise and self-consistent form. 

1.8. Gauss infinitesimal rigidity through the second derivative of 
the volume. Mixed volumes theory basically deals with derivatives of the 
volume bounded by M with respect to infinitesimal deformations rj such 
that drj{TM) C TM. In the proof of the Gauss infinitesimal rigidity that 
can be extracted from [26], one computes the second derivative Vol" under 
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an isogauss deformation in two different ways. One computation yields zero, 
while the other one yields 

Vol" = J 2h det dr] darea, 

M 

the right hand side of which coincides with the left hand side of (|l.ip . The 
operator drj: TM — )• TM turns out to be related to the derivative of -B"^ 
under the deformation r/, which accounts for the properties det dr] < and 
det dr] = dr] = 0. 

1.9. Darboux's wreath of 12 surfaces. The arguments in all three Sub- 
sections 11.41 11.61 and 11.81 end by considering the same equation (|l.ip . As 
just explained, the last two arguments are identical. However, the argument 
sketched in Subsection 11.41 arrives at (11. ip in a different way. It makes use 
of another relation between the Gauss and metric infinitesimal rigidities. 

The translation vector field of an isometric infinitesimal defor- 
mation of M is an isogauss infinitesimal deformation of M*. 
Here M* is the polar dual of the surface M, and the translation vector 
field T is another vector field canonically associated with every isometric 
infinitesimal deformation ^. The above correspondence together with that at 
the end of Subsection 11.71 form a part of the Darboux wreath, see Subsection 
15.31 for more details. In fact, the correspondencies are local, so that M may 
be an open disk embedded in with non-vanishing Gauss curvature. 

In order to compute HE" under the deformation of M, one performs 
the same integration by parts as when Vol" under the deformation r of M* 
is computed. Besides, the deformation r of M* is in some sense polar dual 
to the deformation of M. This suggests that there is a subtle relation 
between the Hilbert-Einstein functional of P and the volume of P* . 

1.10. Polar duality between the volume and the Hilbert-Einstein 
functional. In spherical and hyperbolic geometries there is a straightfor- 
ward relation between the Hilbert-Einstein functional and the volume of the 
dual, see Subsections 16.41 and 16.61 On the other hand, an analog in S'^ and 

of the Gauss infinitesimal rigidity in is the infinitesimal rigidity with 
respect to the third fundamental form of the boundary. Since the third 
fundamental form equals the first fundamental form of the polar dual, the 
metric rigidity is directly related to the Gauss rigidity of the polar dual. As 
a result, variational approaches through HE and Vol to the respective kinds 
of rigidity are equivalent. For details, see Subsection 16.51 

Note that the polar dual to a surface M C is a surface in the de Sitter 
space, see Subsection 16.61 

1.11. Minkowski and Weyl problems. Gauss and metric infinitesimal 
rigidity can be interpreted as "infinitesimal" uniqueness in the Minkowski 
and Weyl problems. Besides, the Minkowski problem for polyhedra and 
for general convex surfaces can be solved by maximizing the functional Vol 
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over some linear subspace in the space of all surfaces. A similar variational 
approach to the Weyl problem was suggested by Blaschke and Herglotz 
[llj : every solution corresponds to a critical point of the Hilbert-Einstein 
functional on the space of all extensions of the given metric g on the sphere 
to a metric g on the ball, cf. Subsections 11.21 and II. 4[ This approach was 
not realized, as the functional is neither concave nor convex, and there is no 
immediate reduction in the spirit of Vol and Minkowski problem above. 

As a modification of Blaschke-Herglotz approach, we suggest to consider 
only extensions of the form Qr described in Subsection 11.21 Further details 
can be found in Subsection 17.41 

1.12. General convex surfaces and other perspectives. The Hilbert- 
Einstein functional has a discrete analog, and most of the constructions 
presented here carry over to polyhedral surfaces, see [31]. A generalization 
to arbitrary convex surfaces suggests itself. In particular, it would be nice 
to find a similar proof of the infinitesimal rigidity of convex surfaces without 
fiat pieces, a notoriously difficult theorem of Pogorelov [331 Chapter IV]. 

Another possible development is to extend our method to Einstein mani- 
folds with boundary, cf. Subsection ll.3| in particular to hyperbolic manifolds 
with smooth convex boundary. (Infinitesimal rigidity of the latter is proved 
by Schlenker in |49j with a different method.) An extension to the case of 
non-regular boundary, as in the previous paragraph, would allow to prove 
the infinitesimal rigidity of convex cores, which could lead to a solution of 
the pleating lamination conjecture. See Section [HI 

1.13. Acknowledgements. I would like to thank Frangois Fillastre and 
Jean-Marc Schlenker for interesting discussions and useful remarks. 

2. Gauss rigidity of smooth convex surfaces 

2.1. Definitions and statement of the theorem. Let M C be a 

smooth surface. 

Definition 2.1. An infinitesimal deformation of a smooth surface M C 
is a smooth vector field along M , i. e. a smooth section of the vector bundle 
TR^Im M. 

By using the natural trivialization of the bundle TR^Ijv/, we can view an 
infinitesimal deformation of M as a map 

(2.1) r/iM^M^. 

The geodesic flow along an infinitesimal deformation r] produces a family of 
smooth maps 

(ft: M ^ M^, 
ipt = exp(tr/). 

Equivalently, if ?y is viewed as the map (12. ip , then we have 

(2.2) ipt{x) = x + tr]{x). 
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We restrict our attention to t € (— for some small positive £, so that all 
ipt are smooth embeddings. Put 

Mt := MM). 

Let Kf-. Mt ^ R he the Gauss curvature of Mt. Denote hy K: M ^ R its 
derivative at t = 0: 

Kf o ifit — K 

(2.3 K := lim * . 

Definition 2.2. An infinitesimal deformation r] is called isogauss, if the 
following two conditions are fulfilled: 

(a) T^M is parallel to T^^^(^^-^Mt for all x € M and for all t G (— e, e); 

(b) k{x) = for all x G M. 

In other words, the flow of an isogauss infinitesimal deformation preserves 
the Gauss map and preserves the Gauss curvature in the first order of t. 

Definition 2.3. An infinitesimal deformation rj of is called trivial, if 

7]{x) = a for all x G M, 

for some a G M^. 

If rj is trivial, then the map ipt is a parallel translation. Hence every 
trivial infinitesimal deformation is isogauss. If a surface has no isogauss 
infinitesimal deformations other than trivial ones, then we call it Gauss 
infinitesimally rigid. 

Theorem 2.4. Let M C R^ be a convex closed surface with everywhere 
positive Gauss curvature. Then M is Gauss infinitesimally rigid 

We were unable to find a reference for Theorem l2.4l but its statement and 
proof are implicitely contained in Hilbert's treatment [26] of Minkowski's 
mixed volumes theory, see also [15] and \28\ Section 2.3]. For the connection 
with the Minkowski problem see discussion in Subsection 17. 2i 

Theorem 12.41 holds for hypersurfaces in R'^ as well, with K interpreted as 
the determinant of the shape operator. 

2.2. Variation of the Gauss curvature. Let ly he a local unit normal 
field to M, and let p be the position vector field on M. Similarly to rj, these 
two vector fields can be viewed as smooth maps 

I/: M ^ R^ 

p: M ^ R^. 

From this point of view, p is the inclusion map p{x) = x. 
Consider the differential 

du: TM R^. 
The map di^ is obviously related to the covariant derivative 

Vi/: TM T\mR^, 
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namely, diy{X) is the image of Vxi^ under the map TR^ which trans- 

lates the starting point of a vector to the origin. We define the shape oper- 
ator B : TM —7- TM as B(X) = Vx^^ and write by an abuse of notation 

dv = B: TM TM. 

Similarly, the differential dp interpreted as the covariant derivative is a map 
from TM to TIjv/M"^. It is easy to see that 

dp = id: TM TM. 

Finally, condition (jaj) from Definition l2.2l is clearly equivalent to drj{X) \\ TM 
for all X, so that we have yet another operator 

dr]-. TM TM. 

Lemma 2.5. Let r] be an infinitesimal deformation of a surface M such 
that the condition (|aj) from Definition 12.21 is satisfied. Then we have 

(2.4) k = -Ktr{d7]). 

Proof. Consider the surface Mt, see (j2.2p . Let vt be the unit normal field 
to Mt (inducing the same co-orientation as v on M), and rjt be the position 
vector field. Condition ([a]) implies I't ° = ^- It follows that 

dut ° dipt = du. 

By taking the determinant, we obtain 

{Kt oift) ■ det(id +tdr]) = K. 

After substituting in (12. 3p and observing that 

daiiid+tdT]) = 1 + tiT{d7]) + t"^ det{dr]) 

we have 

_ (j..oV,.)(l-det(id+id,ri) ^ 

t^o t ^ " 

The lemma is proved. □ 

2.3. Proof of Theorem 12. 4L The proof is based on two lemmas. 

Lemma 2.6. Let t] be an isogauss infinitesimal deformation of M. Assume 
that M has a positive Gauss curvature at x. Then at this point x we have 

(a) det((i?/) < 0; 

(b) the equality dei{dr]) = holds only if di] = 0. 

Proof. On the four-dimensional vector space End(TrM), consider the qua- 
dratic form det. The associated symmetric bilinear form is called the mixed 
determinant, see Subsection lA.ll for more details. As det (id, ^) = ti A for 
all A, equation (12. 4p implies 

(2.5) det(id,dr?) = 0, 

because we have K = and K ^ 0. By differentiating the identity 

{i^,dii) = 
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and using the fact that the connection V is flat, we also obtain 

(2.6) det{JB,dr]) =0, 

where J : TxM — )• TxM is the rotation by ^. Thus the vector drj G End(ra;M) 
is orthogonal to the vectors id and JB with respect to the form det(- , •). 

On the other hand, det takes positive values on both id and JB; the latter 
value is > by our assumption. As the signature of det is (+, +, — , — ) 
and vectors id and JB are linearly independent, this implies that drj lies in 
a two-dimensional subspace of End(T^.M) on which det is negative definite. 
This implies both statements of the lemma. □ 

Definition 2.7. Let M C be an embedded orientahle surface with a 
chosen unit normal field v. The support function 

/i: M ^ M 

sends every point x to the (signed) distance of the tangent plane T^M from 
the coordinate origin in R'^ . The distance is considered positive if the normal 
V points away from the origin. 

Clearly, we have h{x) = {u{x),x). In terms of the position vector field p 
this can be written as 

h = {y,p). 

Lemma 2.8. For every isogauss infinitesimal deformation rj of a closed 
surface M C M.^ we have 

(2.7) j 2hdet{dr]) dareaA/ = 0, 

M 

where h is the support function of M. 

Proof. The proof uses partial integration. We will present it in terms of 
vector- valued differential forms, see Section |A] for more details. 

View a map : M — >■ as an M^-valued differential 0-form. The differ- 
ential dr] is naturally an M^-valued differential 1-form. Wedge product of 
vector- valued forms takes values in the tensor power of M^. For example, 
p A dry A dry is a differential 2-form on M with values in M'^(8iM'^(g)M'^. Consider 
the linear map 

dvol: (g) (g) ^ M, 

sending a triple of vectors to the signed volume of the parallelepiped spanned 
by them. By applying dvol to the coefficient of the form p A drj A drj, we 
obtain a real-valued 2-form 

dvol(p A dr/ A drj) G n^{M). 

At the same time, drj can be viewed as a TM-valued 1-form, see Subsection 
12.21 so that daieaM {drj A drj) is also a 2-form on M. We have 

(2.8) dvol(p A drj A dr]) = h daiea.M{drj A drj) = 2h det{dri) dareajv/ • 
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Due to the Leibniz rule and to d{dri) = (see Subsections IA.4I and lA.Sp 
we have 

(2.9) (i(dvol(p Adr] A drf)) = dvol((ip At] Adrj) + dvol(p A dr] A drj). 
Similarly to (fZH]) . 

dvol{dp A 77 A dr]) = — (i/, rj) dareaM^dp A drj) = — 2(i/, rj) dei{dp, dr]) dareajvf • 
As dp = id, equation (|2.5p (which holds because r] is isogauss) implies 

dvol(dp Ar] A dr]) = 0. 
Thus, integrating (12 .9^ and applying the Stokes formula yields 

j dvol(p Adr] A dr]) = 0. 
M 

Together with (12.80 this implies the statement of the lemma. □ 

Proof of Theorem \2.4\ Let M be a closed surface with everywhere positive 
Gauss curvature, and let r] be an isogauss infinitesimal deformation of M. 
We may assume that the coordinate origin of lies in the region bounded 
by M, which implies h > 0. Together with Lemma 12.61 (jaj) this yields 

J 2hdet{dr]) darea^/ < 0. 

M 

On the other hand, by Lemma [2.81 eg nation (|2.7p holds. Thus we must have 
det{dr]) = everywhere on M. By Lemma 12.61 (jaj) this implies dr] = 0. 
Hence the map r/: M ^ is constant i. e. is a trivial deformation. The 
theorem is proved. □ 

Remark 2.9. Puh the shape operator Bt : TMt TMt back to TM by 
identifying TMt with TM through parallel translation. Then it is easy to 
see that 

Bt = (id +tdr])~^ o B, 

which implies 

B = -dr] o B. 

This equation leads to alternative proofs of equations 1)2.40 and 1)2.60 : 

k = {det By = 2det{B,B) = -2 det{dri o B , B) 
= -2 det{dr], id) ■ det B = - tr{di]) ■ K 

det(JB, dr]) = det( J5, -BB~^) = det( J, -B~^BB~^) ■ det B 

= det(J, (B"^)')-detS = 0, 

where the last equation holds because det{J,A) = for every self-adjoint 
operator A. 
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2.4. Second derivative of the volume. Here we give a geometric inter- 
pretation of the integral j'^^j hdet{dr]) dareaAf that plays a crucial role in the 
proof of Theorem 12.41 

Lemma 2.10. Let P C be the body bounded by a closed surface M , and 
let M — 7- M'^ be the position vector field along M . Then we have 

(2.10) Vol(P) = ^ y dvol(p A dp A dp). 

M 

Proof. We have 

dvol(dp A dp A dp) = 6 

(cf. Lemma lA. lip . Thus 

Vol(P) = - I dvol( dp A dp A dp) = - [ dvol(p A dp A dp), 
6 7 6 J 

P M 

where we applied the Stokes theorem in the last equality. The lemma is 
proved. □ 

Remark 2.11. In a more classical notation, formula ()2.10p reads as 

Vol(P) ~ J ^ dareaM 

M 

and can be obtained similarly by integrating the function A(||x|p) = 6 over 
P and applying the Stokes theorem. 

Let r] be an infinitesimal deformation of M satisfying condition ^ from 
Definition 12.21 Let Mt be the surface defined in Subsection 12.11 and denote 
by Pt the body bounded by Mf. Denote further 

dt^ 



d 

Vol = — 

dt 



Yol{Pt), Vol' 

t=o 



Vol(Pt) 

t=0 



Lemma 2.12. We have 

(2.11) Vol' = ^ y dvol(r/ Adp Adp) = ^ J dvol(p Adr] A dp), 

M M 

(2.12) Vol" = J dvol{rj A dr] A dp) = J d\o\{p A di] A drj) . 

M M 

Proof. Pull back the position vector field pt ■ Mt — ?> to M by the map (ft. 
Retaining the same notation, we have 

Pt=P + ill- 
Lemma 12.101 implies 



Vol(Pt) = ^J dvol(pi A dpt A dpt). 



M 
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By differentiating and using Lemma I A. 121 we obtain 

Vol' = - I dvol(ry A dp A dp) H — / dvol(p Adrj A dp). 
6 7 3 J 

M M 
Integration by parts shows that the integrals on the right hand side are 
equal. This implies equations (12. lip . 

Equations (j2.12p are proved in a similar way. Formula for integration by 
parts appeared already in (12. 9p . □ 



Equations and ([^1^ imply 

Vol" = / 2/idet ((ir/) dareaM- 
Jm 



The proof of Theorem 12.41 given in Subsection 12.31 consists in computing the 
second derivative of Vol in two different ways — the first and the second in- 
tegral in (I2.12P — and then showing that the first integral vanishes (Lemma 
12. Sp while the second one is non-positive and can vanish only if r/ is trivial 
(Lemma 12. 6p . 

2.5. Hessian of the support function. A more traditional way to present 
the above proof is in terms of the support function. 

First, we need to change the setup. Instead of considering a closed convex 
surface M C ffi^, we consider a smooth embedding 

§2 ^ M^. 

An infinitesimal deformation 

rj: 

determines a family of embeddings 

(ft = ip + trj. 

Finally, the position vector field p coincides with the map (p: 

p = ip. 

If M = f{S'^) has everywhere positive Gauss curvature, then the Gauss 
map from M to is one-to-one. So we can make the following important 
assumption: ip is the inverse of the Gauss map. In other words, we view 
as a unit sphere centered at € M'^, and the map (p sends every x € to a 
point on M with outward unit normal x. 

The support function is now also being viewed as a function on S^, namely 

(2.13) h = {i,p), 
where t: — )■ is the inclusion map. 

Lemma 2.13. The position vector of a Gauss image parametrized surface 
is given by 

(2.14) p = hi + Vh, 
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where V is the Levi-Civita connection on S^. 

Proof. By differentiating (I2.13P in the direction of a vector X G T^^^S^, we 
obtain 

X{h) = (Vxi,p{x)) + (^i{x),Vxp) = {X,p{x)) + {x,d^{X)). 

As d(p{X) € T^{x)M II TxS"^, the last summand vanishes and we have 

{X,Vh) = {X,p{x)). 
It follows that V/i is the orthogonal projection of p{x) to TxS'^, that is 

V/i = p{x) — {x,p{x))x = p{x) — h{x)x, 
and the lemma is proved. □ 

The linear map dp : TS^ TM'^ is related to the shape operator. Namely, 
as p = (p is the inverse of the Gauss map, dp postcomposed with the parallel 
translation of T^{x)M to T^S^ ig the pullback of B'^ to S^: 

(2.15) dp = ip*{B-^). 

Lemma 2.14. Let 93: — > M 6e the inverse of the Gauss map. Then 
the pullback of the inverse of the shape operator expresses in terms of the 
support function h: §^ — ?■ M as 

(2.16) ip*{B^^) = hid + Ressh, 

where Hess/i: X 1— ?• Vx(V/i) is the (1, 1)-Hessian of function h. 

Proof. By differentiating (I2.14p and using (j2.15p we obtain at every point 
a; € §^ the equation 

ip*{B~^){X) = X{h)x + hX + Vx(V/i). 

As the left hand side lies in T^-S^, we have 

X{h)x + hX + Vx{yh) = T{X{h)x + hX + Vx{^h)) 

= /iX + T(Vx(V/i)) = + Vx(V/i), 

where T : T^M.^ — >■ TJ^S^ is the orthogonal projection. This proves the lemma. 

□ 

Lemma 2.15. The volume of the body P bounded by M expresses in terms 
of the support function as 

Vol(P) ~ 2 y ^det(/i id + Hess/i) dareag2 . 

Proof. This follows from 

Vol(P) ~ J hoLp~^ dareaM ~ J ^det(i?~^) dareag2 

M S2 

and from equation (|2.16p . □ 
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By equation ()2.14p , the position vector field and the support function are 
related in a linear manner. Therefore every infinitesimal deformation can 
be expressed in terms of the support function by the same formula: 

rj = kt + Vk, 

where : — )■ M is an arbitrary smooth function. It follows that formulas 
()2.12p can be rewritten as 

(2.17) Vol" = J A;det(S-\(B-^)') darea§2 = J /idet((S-^)') darea§2, 



or 



Vol" = J A; det(A; id + Hess fc, /i id + Hess /i) darea§2 
hdet{h id + Hess /i) darea§2 . 



§2 

The proof of rigidity goes similarly to Subsection 12. 31 The assumption K = 
implies 

det(i?-\(i3-i)-) = ^(det(B-i))- =0, 

so that the first integrand in (|2.17|) vanishes. Non-positivity of the second 
integrand is proved simpler than in Lemma 12. 6t as is self-adjoint, it 
suffices to prove its det-orthogonality to one operator with positive deter- 
minant, and we can just use for this the last equation (cf. Corollarv lA.6p . 



Remark 2.16. The Hessian of the support function was used by Hilbert [26] 
to provide a basis of mixed volumes theory for bodies with smooth boundary 
established earlier by Minkowski for general convex bodies. The argument 
outlined in this section serves as a lemma in a proof of the Alexandrov- 
Fenchel inequality. A recent generalization of the latter is obtained in [23]. 



3. Metric rigidity of smooth convex surfaces 

3.1. The Liebmann-Blaschke theorem. Let M C be a smooth sur- 
face, and let 

^: M ^ M^ 

be a vector field along M. Similarly to Subsection 12.11 consider for small t 
the family of surfaces 

Mt = iftiM), 
(pt{x) = x + 

Let gt be the induced Riemannian metric on Mf. We write go = g. Put 

f*t9t - g 



g = lim 



t-5>0 t 
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Definition 3.1. An infinitesimal deformation ^ of a surface M C M'^ is 
called isometric, if g = 0. 

Definition 3.2. An infinitesimal deformation ^ of a surface M C M.^ is 
called trivial, if ^ is the restriction of an infinitesimal isometry (Killing 
vector field) ofM?: 

^(x) = a X X + 6 

for some a,b £ M^, with x meaning the cross product of vectors in M"^. 

As the Lie derivative of can^a along a Killing vector field vanishes, every 
trivial infinitesimal deformation is isometric. If a surface has no isometric in- 
finitesimal deformations other than trivial ones, then it is called (metrically) 
infinitesimally rigid. 

Theorem 3.3 (Liebmann-Blaschke-Weyl) . Let M C ffi^ be a convex closed 
surface with everywhere positive Gauss curvature. Then M is infinitesimally 
rigid. 

This theorem was proved by Liebmann in |39] under the analyticity as- 
sumption, and later by Blaschke and Weyl for C^-surfaces. A modern ver- 
sion of Blaschke- Weyl's proof is given in Subsection 13.41 the background of 
this proof is explained in Subsection 15. 1[ 

3.2. Rotation and translation vector fields of an isometric infini- 
tesimal deformation. Recall from Subsection 12.21 that we denote by 

p: M -^M.^ 

the position vector field p{x) = x, and that for any vector field, say ^, we 
view its differential (or covariant derivative) as a map 

d^: TM Ml 

In particular, dp{X) = X for every X € TM. 
The following lemma is due to Darboux |21j . 

Lemma 3.4. Let ^ be an isometric infinitesimal deformation of a surface 
M. Then there exists a unique pair {rj,T) of vector fields along M such that 

^ = ri X p + T 

and moreover 

(3.1) d^ = r]X dp, 

(3.2) dr = px dr]. 

Proof. The condition ^ = is easily seen to be equivalent to 

{daX),Y) + {X,daY)) = 

for all X, y € TM. This implies that d^: TM — t- M.^ has a unique exten- 
sion to a skew-symmetric operator yl: R'^ — > M'^. As every skew-symmetric 
operator in is the cross product with a fixed vector, this defines a vector 
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field 7/ : M — )• M'^ satisfying condition (13. ID . If we put t = ^ — rj x p, then 
the condition (|3.2p is automatically fulfilled. The lemma is proved. □ 

Intuitively: an isometric infinitesimal deformation ^ moves every tan- 
gent plane TxM as a rigid body. Thus there exists a first-order approx- 
imation of ^ in a neigborhood of every point xq by a Killing vector field 
X !->■ r]{xQ) X X + t(xo). 

Definition 3.5. Vector fields rj and r are called the rotation and translation 
vector fields of an isometric infinitesimal deformation ^. 

Note that the rotation field is invariant under translations of surface M 
while the translation field is not. Namely, if M is translated by a vector 
a € M'^ and ^ kept unchanged, then the corresponding vector fields are 

p = p + a, rj' = r], t' = t + rj x a. 

Lemma 3.6. We have dr]{X) € TM for all X G TM. 

Proof. This can be proved by taking the differential of (13. 1|) (or computing 
second covariant derivatives VxVy^ and Vy VxO- ^ 

Lemma 3.7. An isometric infinitesimal deformation is trivial if and only 
if its rotation vector field is constant. 

Proof. The rotation vector field of a trivial isometric infinitesimal deforma- 
tion is constant by Definition 13.21 and Lemma 13. 4[ 

In the opposite direction, if i] = const, then dr] = 0, and by (j3.2p dr = 0. 
Hence r = const. By definition, if both rj and r are constant, then ^ is 
trivial. □ 

3.3. Variation of the shape operator. Let v. M ^ M.^ he a local unit 
normal field to M, and let z/j : Mt — )■ M.^ be the corresponding local unit 
normal field on Mt. Consider the shape operators B = du and 

Bt = dvt: TMt TMt. 

Denote 

B = lim-i-^^ . 

t^o t 

Lemma 3.8. For every isometric infinitesimal deformation ^, the operator 
B : TM TM is self-adjoint. 

Proof. The self-adjointness of Bt with respect to gt implies 

[^*tgt)mBt){X),Y) = {^lgt){X, {^*tBt){Y)) 
for all X, y € TM. By differentiating with respect to t, we obtain 

g{B{X),Y) + g{B{X), Y) = g{X, B{Y)) + g{X, B{Y)). 
As ^ is isometric, we have g = 0. This implies the lemma. □ 
The next lemma relates operators dr] and dr with B. 
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Lemma 3.9. Let ^ be an isometric infinitesimal deformation of a surface 
M C M^ , and let r] and r be its rotation and translation vector fields. Then 
we have 

(3.3) dr] = JB, 

where J: TM TM is rotation by 7r/2. We also have 

(3.4) TodT = -hB, 

where T : ^ TM is an orthogonal projection, and h: M ^ M. is the 
support function, see Definition \2. 7| 

Proof. Denote 

V := lim ■ 



t->o t 

By differentiating equations [jt'tlP = 1 and {vt, dipt{X)) = and using (j3.ip . 
we obtain 

i> = 77 X 

(Tfiis is intuitively clear: if a tangent plane to M is rotated by r], then the 
normal is also rotated by r].) Thus we have 

(3.5) dv = dr] ^ V + r] ^ dv = — J o dr] + rj x B, 

where we used that dr]{TM) C TM, see Lemma 13.61 
On the other hand, 

d{yt ° ft) = dvt o dipt = dipt o LptBt. 
Differentiating with respect to t yields 

(3.6) du = d£,oB + dpoB = r]xB + B, 

compare [511 Equation (0)]. Equating the right hand sides of (13. 5p and ()3.6p 
yields equation ()3.3p . 

Now let us prove Equation ()3.4p . We have 

dr = p X dr] = [hv + Tp) x dr] = hv x dr] + cv 
for some c G M, because dT]{X) € TM. Hence, due to (|3.3p 

T o dr = hu X dr] = hJ o di] = hJ^B = —hB. 
The lemma is proved. □ 

3.4. Blaschke's proof of Theorem 13.31 Here we give a coordinate-free 
version of the proof from jlO] (see also [531 Chapter 12]). The main novelty 
is the interpretation of the operator dr] : TM TM through the variation 
of the shape operator, see Lemma \3M 

Blaschke 's proof of Theorem \3.3l Let M C be a closed surface with pos- 
itive Gauss curvature, and let ^ be an isometric infinitesimal deformation of 
M. Let r] be the rotation vector field of ^, see Subsection 13.21 By Lemma 
we have 

dr] = JB: TM TM. 
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This imphes two remarkable properties of the operator dr]: first, 

(3.7) tr(dr/) = 0, 

as tr(Jj4) = for every self-adjoint operator A, and B is self-adjoint by 
Lemma 13.81 and second 

(3.8) det(d?7) < 0, while det(d7?) = only if drj = 0, 

which holds by Corollary IA.6I as det B = K > hy assumption on M and 
(deti?)' = K = because ^ is isometric. 

Recall that p: M — >■ denotes the position vector field so that we have 
dp = id: TM TM. Consider the differential 1-form 

(3.9) dvol(p A ?? A di]) G 0^(M), 

see Subsections lA. 31 — lA.SI or a brief explanation in the proof of Lemma [2.81 
We have 

d{dvol{p A 7] A di])) = dvol{dp Ar] A dr]) + dvol(p A dr] A dr]). 
The first summand on the right hand side vanishes due to 

dp A dr] = det(id, dr]) dareajw = - tT{dr]) darea^ = 0, 

where we used (j3.7p in the last step. The integrand in the second summand 
equals 2hdet{dr]), where h is the support function of the surface M. Thus 
Stokes' theorem implies 

(3.10) j 2hdet{dr]) dareaM = 0. 

M 

Without loss of generality we may assume that the origin of lies in the 
region bounded by M, so that h > 0. Together with (|3.8p . this implies 
dr] = 0. Thus r] is constant, and by Lemma 13.71 the deformation ^ is trivial. 

□ 

Subsection 15.11 tells about the origin of this proof. We'll now give another 
proof that looks more complicated at first but allows a variational interpre- 
tation similar to that given in Subsection 12.41 to the proof of Theorem 12. 4[ 

3.5. Another proof of Theorem 13.31 Let ly be the field of outward unit 
normals to M. Introduce a new vector field along M 

Consider the differential 1-form 

dvol(7/; A r A dr) € J^^(M) 

and its differential 

d(dvol(^ A T A dr)) = dvol((iV At Adr) + dvol(V' Adr A dr). 
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By Stokes' theorem, 

(3.11) j dvol(r Adip Adr) = j dvol(V' A dr A dr). 
M M 

Lemma 3.10. We have 

dvol(T A A dr) = 0. 
Proof. First of all, let us show that 

(dr,p) = 0= (dV^.,p). 
The first equality follows from dr = p x drj. For the second one, compute 

{di^{X),p) =X{'iP,p) -{^,X). 

The right hand side vanishes as ip -L X and {ip,p) = 1. Thus the left hand 
side vanishes too, and the claim is proved. 
It follows that 

(3.12) dvol(T A dip A dr) = -^^-^ darea„± (# A dr), 

\\P\\ ^ 

where darea^x is the area form in the orthogonal complement to vector p. 
Substitute dr = p x dr]. A simple computation shows that 

dareapx(V A (p x dr])){X,Y) = \\p\\ {{ij{X),dr]{Y)) - {i^{Y),dr,{X))) . 

By using (13. 3p . we compute 

{d'ip{X),dT]{Y)) = {h-^B{X) + X{h-^)u,JB{Y)) 

= h-^{B{X),JB{Y)) = -h~^dam&M{B{X),B{Y)). 

It follows that 

llpll 

(3.13) darea„±(dV A dr) = — dareaAf(-B, B) = 0, 

^ h 

because 2det(i?,-B) = (detS)' = iiT = for an isometric infinitesimal de- 
formation. Substitution in (|3.12p proves the lemma. □ 

Lemma 3.11. We have 

dvol(/i~^j^ A dr A dr) = 2h det B darea^./ . 

Proof. Recall that T : — )■ TM denotes the orthogonal projection (thus T 
sends a vector in TM'^Ia/ to its tangential component). We have 

dvol(/i~^i^ A dr A dr) = dareaA/(T o dr, T o dr) 

= 2h~^ det(T o dr) = 2/i~^ det{-hB) = 2h det B, 

where we used equation (13. 4p . The lemma is proved. □ 
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By substituting the results of Lemmas 13.101 and 13.111 in ()3.1ip , we obtain 
(3.14) J 2h det B dareaAf = 0. 

M 

The end of the proof is the same as in Subsection 13. 4t we may assume 
h > 0, at the same time by Corollary IA.6I we have det-B < 0; thus we have 
det-B = which again by Corollary I A. 61 implies B = 0. As drj = JB, it 
follows that the rotation vector field is constant, thus by Lemma 13.71 the 
deformation ^ is trivial. 

3.6. Hessian of the squared distance function. Here we rewrite the 
proof from the previous subsection in other terms. For a surface M C M^, 
consider the function 



/: M 



Lemma 3.12. We have 



\/f = p- hu, 

where p: M — )■ M"^ is the position vector field, and h: M — t- M is the support 
function relative to the unit normal field u. 

Proof. Clearly, V/ = p. Thus we have 

V/ = Tp = p — {u,p)v = p — hv. 

□ 

Lemma 3.13. We have 

(3.15) Hess/ = id-/i5, 

where Hess/: TM — > TM , X i— )• VxV/ is the (1, 1)-Hessian of function f. 
Proof. By using Lemma 13.121 we obtain 

VxV/ = T (VxV/) = T (Vx{p - hv) 

= T{X - X{h)u - hB{X)) = X - hB{X), 
which proves the lemma. □ 

Let ^ be an infinitesimal deformation of M, and let Mt = <pt{M) be 
the geodesic flow of M along ^, see Subsection 13.11 Put ft : Mt — s- M, 
ft{x) = ||x|p/2 and 

/ = limA^^i^. 

t^o t 

Lemma 3.14. //^ is an isometric infinitesimal deformation, then we have 

Hess / = -ijiB + hB), 
where B is defined as in Subsection \3.3[ 
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Proof. The isometricity of implies 

Hess / = (Hess/)', 

where (Hess/)' is defined similarly to B. Now the lemma follows by differ- 
entiating equation (j3.15p . □ 

Consider the differential 1-form 

dareaM(V/ A^) G n^{M). 

Here V/ is viewed TM-valued 0-form, and B TM-valued 1-form 
on M. By Subsection lA. 41 we have 

(3.16) d(dareaAf (V/ A B)) = dareaM(Hess f A B) + dareaM(V/ A (fB), 

where : Q}{M,TM) n'^{M,TM) is the exterior differential associated 
with Levi-Civita connection V on M. Isometricity of deformation ^ and the 
Codazzi-Mainardi equations imply 

d^B = {d^By = 0, 

so that the second summand on the right hand side of (j3.16p vanishes. 
Further, by Lemma [3. 141 and because of 2det(i?,-B) = {det B)' = K = we 
have 

dareaM(Hess/ A^) = -2{h det{B , B) + h det B) = -2hdetB. 

By integrating ()3.16p and applying Stokes' theorem we arrive to ()3.14p . 

Remark 3.15. A similar argument is used in the proof of |36l Lemma 3.1] 
that deals with infinitesimal rigidity in the Minkowski space. 

The following lemma allows to relate the above argument to the argument 
in Subsection! 



Lemma 3.16. Let t be the translation vector field of an isometric infini- 
tesimal deformation ^. Then we have 

V/ = Tr. 

Proof. By differentiating the equation ft = ||pt|p/2, we obtain / = {p,S,)- 
Hence for all X € TM we have 

{X, V/) = Vxf = {X, + {P, v^X) = {X,^ + px 7?) = {X, r), 

where we used equations from Lemma 13.41 This implies the statement of 
the lemma. □ 



Together with equation (j3.4p this implies that the differential 1-forms 
used here and in Subsection 13.51 are closely related: 

dareaAf(V/ A B) = dvol{u A Tr A {h^^T o dr)) = dvol(/i~^z> A r A dr). 
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4. Metric rigidity and the Hilbert-Einstein functional 

In this section we provide a variational interpretation of the proof of 
Theorem 13.31 given in Subsections 13.51 and 13.61 This interpretation does 
not simphfy the arguments, rather conversely. But it supports the ideas 
suggested in Section [8] concerning approaches to the Weyl problem and to 
the pleating lamination conjecture. 

4.1. Reduction to warped product deformations. Let M C be a 

smooth convex closed surface such that the coordinate origin lies in the 
interior of the body P bounded by M. Instead of deforming the embedding 
of M into M^, we will deform the metric inside P in the following way. Think 
of P as composed of thin pyramids with apex at and bases on M; vary the 
lengths of lateral edges of the pyramids while leaving the base edge lengths 
constant and thus preserving the intrinsic metric of M. 

If the lengths of lateral edges are given by a smooth function r: M — > R, 
then this construction gives a smooth metric 5v on P \ {0}, called a warped 
product metric. For some functions r, the metric gr is Euclidean, for example 
if r is the distance from some interior point a G P (if we are just moving 
the origin inside P, so to say). The key point is that the surface M is 
infinitesimally rigid if and only if no variations of r besides those mentioned 
above leave the metric gy Euclidean in the first order. The goal of this 
subsection is to explain this equivalence and to state a reformulation of 
Theorem 13.31 

Consider the radial projection 

. y 
\\y\\ 

and denote by ip^ its inverse. 

Definition 4.1. The distance function of a surface M is 

ro: ^M+, 
ro(x) = ||99o(2;)||- 

It follows that the surface M is the image of an embedding 

(4.1) Lfoix) = roix) ■ X. 

Lemma 4.2. Consider the diffeomorphism 

F: M+ X §2 ^ \ {0}, 

(/9, x) I-)- pX. 
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Then the puUback of the canonical Euclidean metric on M'^ \ {0} to M+ x §^ 
by the map F is given by 

F*{can) = dp^ + {g - dro (E> dro), 

where g = ipQ{can) is the metric induced on by the embedding (j4.ip . 
Proof. We know that F*{can) = dp^ + p^cangi. Thus we have to show that 

can§2 = ^(g — dro ® dro), 

^0 

or, equivalently, g = rQcan§2 + dro <Si dro- The last equation follows easily 
by taking the differential of (j4.ip . □ 

Lemma 14.21 motivates the following construction. Let g be a Riemannian 
metric on , and let r : — > be a smooth function such that 

(4.2) ||Vr||g < 1 

everywhere on S^. Associate with {g,r) a Riemannian metric 

(4.3) gr = dp^ + {g-dr® dr) 
on 

M+ X §2. (Condition (g^]) ensures that the symmetric bilinear form 
g — dr ® dr is positive definite.) It is easy to see that the map 

(fr: §2 ^ ]R+ X 

(4.4) ipr{x) = {r{x),x) 

embeds {E>'^,g) isometrically into (1R+ x S^,^^)- 

Assume that the metric ()4.3p is flat, i. e. has a vanishing curvature tensor. 
Then (see [52], |59| Chapter 2.3]) the Riemannian manifold (M_|_ x S'^,gr) is 
locally isometric to M'^, moreover there is an isometry IR+ x M'^ \ {0}, 

since the source space is simply-connected and the metric gr is complete. 
It follows that the map (14. 4p composed with this isometry is an isometric 
embedding of (S^,^') into M.^. 

In particular, if the metric g is induced by an embedding ipo'. S'^ — >■ 
and ro{x) = \\ipo{x)\\, then the metric gro is flat. By the previous paragraph, 
embedding (po is determined by rg up to an isometry of flxing 0. 

Remark 4.3. The embedding ipo from the previous paragraph need not be 
the inverse of the radial projection. Indeed, if the metric gr is flat, then so 
is the metric determined by g' = ij)*g and r' = r oip for any diffeomorphism 
■0: — >■ §2. The distinctive feature of the parametrization (|4.ip is that 
the corresponding isometry (R_|_ x 'SP',gr) — >■ (M^ \ {0}, can) is the map ^> in 
Lemma I4.2[ 



Consider a smooth function s : — )• M and put 

rt=ro + ts. 
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As rg satisfies condition ()4.2p . rt also satisfies ()4.2p for all sufficiently small t. 
Let i?t be the curvature tensor of the metric on x S^; denote 

i? = lim.^*"^° 



t-5>0 t 

Definition 4.4. ^ function s : — )■ M is called a curvature preserving 
infinitesimal deformation of r^, if R = 0. 

Definition 4.5. A function s: — ?• M is called a trivial infinitesimal de- 
formation o/ro, i/ there exists an a gM.^ such that 

(4.5) = 

\ ll¥'o(a;)|| 

As the embedding tpQ is determined by tq up to an isometry of fixing 
0, the class of trivial deformations is well-defined. 

Lemma 4.6. If s is trivial in the sense of Defitinion\4-5\ then s is curvature 



preserving in the sense of Definition [ 

Proof. Let s be as in ()4.5p . Consider the function 

r^x) = 11970(3;) + ta\\. 

As is the distance function of the surface M translated by the vector —ta, 
the metric Ijr'f is fiat. But since r'^ — rt = o{t) in the C^-norm, the metric 
Qr^ is fiat in the first order. Thus R = 0, and s is curvature preserving. □ 

Now we can state the main result of this subsection. 

Theorem 4.7. A smooth convex closed surface M is infinitesimally rigid if 
and only if every curvature preserving deformation of its distance function 
is trivial. 

Proof. The theorem is proved by establishing a correspondence between iso- 
metric deformations ^ of M and curvature preserving deformations s of the 
distance function tq so that trivial deformations correspond to trivial ones. 

Let ^ : ^ be a vector field along an embedding ipQ-. — ?> M^. Define 
the function s: ^ M as 

ipQ{x) 



(4.6) s{x) = U{x) 

\ \m[.x)\\ 

We claim that if ^ is an isometric infinitesimal deformation, then s is cur- 
vature preserving. The proof is similar to that of Lemma 14.61 Consider the 
embedding 994 = + ^C- Let r[ = \\^t\\ be its distance function, and let 
be the metric induced on by tpt. It follows that the metric on M+ x 
obtained by substituting in (|4.3|) g^ for g and r[ for r is flat. As r[ — rt = o{t), 
and gt — 9 = o{t) in the C^-norm, the metric is fiat in the first order. 

Now let s be such that = 0. By [161 Proposition 3], an ifninitesimal 
deformation /i of a fiat Riemannian metric g leaving the curvature zero in 
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the first order is locally induced by an infinitesimal diffeomorphism; i. e. h is 
the Lie derivative of g along a vector field rj. As M+ x is simply-connected, 
the local vector field rj can be extended to a global one, so that we have 



d_ 

dt 



9rt — '-TI 9r(, ■ 

t=0 



Note that rj is defined uniquely up to adding a Killing vector field with 
respect to the flat metric gr^- It can be shown that the limit 

r?(0) := lim ri{p,x) 
p— >o 

exists. Add to rj a parallel translation so that i]{0) = and put 

^{x) = r]{ro{x),x). 

It is easy to see that ^ is an isometric infinitesimal deformation of the surface 

M = {(ro(x),x)} C M+ X §2. 

Besides, the variation s of the distance of the point {rQ{x),x) from the 
singular point p = is related to ^ through the formula (|4.6|) . (This is due 
to the fact that 77(0) = and that the lines M_|_ x {x} are geodesies in the 
metric gr-) It follows that the map s 1-^ ^ thus obtained is the inverse of the 
map constructed in the previous paragraph. Note that ^ is well-defined up 
to adding a rotation around 0. 

Trivial deformations are sent to trivial ones; in particular, if s satisfies 
(j4.5p . then we can put 

ro(x) 

which results in = a. □ 

The curvature tensor of the metric gr is completely determined by sec- 
tional curvatures in some small set of planes, see Section [Bj For example, it 
suffices to take all planes tangent to the surface 

(4.7) Mr = {{rix),x)}. 

Definition 4.8. Denote by sec(r)(x) the sectional curvature of the Rie- 
mannian manifold (M+ x S'^,gr) in the plane T(^y.{x),x)^r- We will usually 
omit the argument r of sec. 

Let s = r be an arbitrary infinitesimal deformation of the function r. 
Denote by sec' the derivative of sec in the direction r: 

., , sec(r -|- tr)(a;) — sec(r)(x) 
sec (x) = lim . 

Corollary 4.9. A smooth convex closed surface M = ipoiS'^) is infinitesi- 
mally rigid if and only if every infinitesimal deformation r of its distance 
function rg such that sec' = is trivial. 



RIGIDITY OF SURFACES AND THE HILBERT-EINSTEIN FUNCTIONAL II 27 



4.2. Hessian of the squared distance function in a warped product. 

Consider the function 

/: R+ X §2 ^ M, 

and denote by Hess/ its Hessian with respect to the metric gr in (|4.3|) . 
U^sfiX) = VxV/. 

Lemma 4.10. 

Hess/ = id 

Proof. We have (see [l3l Chapter 2, Section 1.3]) 

gr (ltes/(X),y) = ^£^^griX,Y). 

The flow Ft on x generated by V/ scales the /9-coordinate by e*. It 
follows that 

V f 9r — 2(7r • 

Thus Hess/(X) = X for all X, and the lemma is proved. □ 

We now introduce several vector fields along the surface C x 
given by (j4.7p . First, there is the field of outward unit normals v. Second, 
let dp be the radial unit vector field on x S^. The vector field pdp 
generalizes the position vector field p that we considered in the case of a 
surface M C M"^. Since dp has a unit norm and is orthogonal to TS^ with 
respect to gr, it is also the gradient field of the function p: {p,x) i— t- p. It 
follows that 

(4.8) V/ = pVp = pdp. 
Denote the restriction of the function p to Mr by r: 

r: Mr ^R, 

r = P\Mr- 

(This is an abuse of notation, as earlier we denoted by r the function ip~^ o p 
on S^.) Let us introduce two auxiliary functions on Mr. 

Definition 4.11. Denote by a: Mr [0, f) the angle between vectors dp 
and V, and define h: Mr — )• M_|_ as h = rcosa. 

The function h generalizes the support function defined in the case of a 
surface M C M^. 

Lemma 4.12. Let V be the covariant derivative on Mr associated with the 
metric g = gr\Mr- Let Hess / : X i— > VxV/ be the Hessian of f\M^. Then 
we have 

(4.9) Hess/ = id-/iB, 
where B: X ^ Vx^^ is the shape operator on Mr. 
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2 

Figure 1. The gradients of / = ^. 



Proof. By gS]) and Definition KTT\ we have V/ = T(V/) = V/ - hu. 
Further, 

VxV/ = T(VxV/) = T(Vx(V/ - hu)) 

= T(Ifes/(X) - X{h)iy - hB{X)) = X - hB{X), 

which proves the lemma. □ 

4.3. The Hilbert-Einstein functional. Denote 

Pr = {{p,x) eR+x§^ \ p< r{x)}. 

Definition 4.13. The Hilbert-Einstein functional of the Riemannian man- 
ifold with boundary {Pr,gr) is defined as 

(4.10) RE{r) = - scaldvol+2/ darea, 

where seal is the scalar curvature of the metric Ijr, and H = ^tri? is the 
total mean curvature of the surface Mj. = dP^ . 

The manifold D^. is not compact (as we exclude the point p = 0). However 
the first integral in (j4.10p converges; this follows from the next lemma that 
expresses it as an integral over Mj.. 

Recall that sec(a;) denotes the sectional curvature of Qr in the plane tan- 
gent to Mr at {r{x),x). 



SGC 

seal dvol = 21 r darea, 



Lemma 4.14. 

IPr J Mr COS a 

Proof Put g = S^f^, so that we have 

5V = dp^ + p^g. 



Then dvol = p^ • dp f\ dareag and 



seal dvol = / darea^ / p^ seal dp. 



2 
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Equations ()B.3|1 and (1B.4|) imply 



scal(/3,x) = 2sec(p^^)(9^ ) = 2 • 

Substituting this in the previous equation yields 

(4.11) / seal dvol = 2 / r • sec(i ^.^(Sj") dareas . 

We now want to rewrite this as an integral over Mr- First, note that the 
Jacobian of the radial projection vr: {1} x — > Mr equals so that 

cos a 

dareag = — — ■ vr (darea^j. 
And secondly, again by ()B.4p and (|B.3p . we have 

sec(^i,x){dp) = r{xf ■ sec(^r{x),x){dp) = r 
By substituting both equations in (j4.1ip . we obtain 



cos^ a 



T T , r, / 2 sec cos a , / sec , 
seal dvol = 2 r ■ r — • — -— darea = 2 1 r darea, 



IPr J Mr cos- a J Mr cos a 

and the lemma is proved. □ 

Theorem 4.15. 

HE(r) = / h{K + det B) darea, 

J Mr 

where K is the Gauss curvature of Mr, and h is as in Definition \4-ll ■ 
Proof. Consider the differential 2-form 

darea(Hess f A B) 

on Mr- Here the linear operators Hess / and B are viewed as TMj.-valued 
1-forms; their wedge product and operation of darea on it are defined in 
Subsection IA.3[ We have 

Hess/ = V(V/) = (i^(V/), 

where is the exterior derivative on TM^-valued forms. Thus by Lemmas 
IA.15I and IA.14I we have 

(i(darea(V/ A B)) = darea(Hess/ A B) + darea(V/ A d^B). 

Stokes' theorem implies 

(4.12) / darea(Hess f A B) + / darea(V/ A d^ B) = 0. 

J Mr J Mr 

With the help of Lemmas lA.llI and 14.121 the first integrand in (j4.12p can 
be computed as 

(4.13) darea(Hess f A B) = 2{H - hdet B) darea . 
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To compute the second integrand in (14.120 . observe that 

cfBiX,Y) = VxiBiY))-VYiBiX))-Bi[X,Y]) 
= VxVyZ/ — Vy VxJ^ — V[x,y]l^ 
= T(VxVyl/ - VyVxl' - V[x,y]l') 

= T(i?(x,y)i/). 

By (fR2]) we have 

R{X, Y)u = sec(9^) • dvol(9p, X, Y) ■ {v x dp) G TMr. 

Note that 

dvol(9p, X, Y) = cos a ■ darea(X, Y), u x dp = J(Vr), 
where J: TMr TM^ is a rotation by ^. Besides, by ()B.3P we have 

sec{d. ) 



cos^ a 



As a result, we have 



SGC 

(4.14) d^B = J(Vr) darea, 

cos a 

so that 

(4.15) darea(V/ A d^B) = (V/, Vr)^^ darea . 

cos a 

By substituting (|4.13|) and (|4.15|) in (|4.1'2|) and using the identity 
(V/, Vr) = r|| Vr Ip = r sin^ a, 

we obtain 

/ H darea = [ h det B darea — [ r sin^ a darea . 

J Mr J Mr 2 J Mr COS Q 

Together with Lemma |4. 141 this imphes 

HE(r) = / (r h 2/i det i? — r sin^ a ) darea 

J]\.f^ V cos a cos a/ 

= I {h sec +2h det B) darea = / h{K + det B) darea . 

J Mr J Mr 

Here we used the identity h = r cos a and the Gauss equation 

(4.16) K = sec + detB. 

The theorem is proved. □ 

Remark 4.16. For a surface M in the same argument yields one of the 
Minkowski formulas: 



/ H darea = / hK darea, 
Jm Jm 
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see [53^ Chapter 12]. The other Minkowski formula 



Area(M) = hH darea 
Jm 



also holds in our more general situation and can be proved by integrating 
the 2-form darea(Hess / A id). 

4.4. First derivative of the Hilbert-Einstein functional. The Hilbert- 
Einstein functional is a differentiable map from C°°(S2) to M. Denote by 
HE' the derivative of HE in the direction of r G C°°(S^). 

Theorem 4.17. 

(4.17) HE' = / r darea 

J Mr COS a 

Proof. Similar to the proof of Theorem 14.151 we integrate by parts the dif- 
ferential 2-form 

darea(Hess f A B). 

We have 

(4.18) / darea(Hess f A B) + / darea(V/ A B) = 0. 

J Mr J Mr 

As the metric g on Mr does not depend on r (rather, one should speak 
about the metric induced on by (fr- — )• Mr), we have V = 0, and hence 

Hess / = (Hess /)' = -iiB - hB. 

Thus the first integrand in (I4.18P equals 

darea(Hess f A B) = -2{h AeiB + h det(S, B)) darea . 

By Lemma lA.51 we have 2det(-B,-B) = (detS)'. Besides, due to the con- 
stancy of the metric g we have iiT = 0, so that the Gauss equation (j4.16p 
implies (det-B)' = — sec'. As a result, we obtain 

(4.19) darea(Hess / A -B) = (-2/i det i? + /i sec') darea . 
To compute the second integrand in (14.180 . substitute ()4.14p : 

darea(V/ A d^S) = (V/,Vr) darea. 

cos a 

An easy computation yields 

(4.20) (V/, Vr) = r - AcoscK, 
so that we have 

(4.21) darea(V/ A (f B) = {j^^ - h sec) darea . 

By substituting ()i39]) and ()i:2T]) in ()i38]) . we obtain 

sec ■ \ 

-2h det B + h sec' +r h sec ) darea = 0. 

Mr ^ cosa / 
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On the other hand, by differentiating the formula from Theorem 14. 151 we get 
HE" = / (h{K + det B) + h{det By) davea 

h sec +2h det B — h sec" ) darea 



I Mr 

By substituting this in the previous formula, we obtain (j4.17p . The theorem 
is proved. □ 

Remark 4.18. By differentiating the formula in Lemma [4.14l and combining 
the result with (I4.17p . we obtain 

ij darea = — / r( \ darea. 

Mr '^JMr ^ COS a J 

The same can be proved by integrating darea(Hess / Ai?). It follows that the 
derivative of the total mean curvature vanishes if {■^^) vanishes at every 
point. This happens in particular for sec = = sec", that is for isometric 
infinitesimal deformations of a surface in W^. See [U l47l [5T| [5]. 

Remark 4.19. Theorem 14.171 can also be derived from the general for- 
mula for the derivative of the Hilbert-Einstein functional on a manifold 
with boundary, see e. g. (TJ Equation (2.9)], provided that care is taken 
of the singularity at p = 0. For this, one has to estimate the decay of the 
derivative of scalar curvature at p 0, which is easily done. 

4.5. Second derivative of HE and the proof of Theorem 13.31 Denote 
by HE"" the second derivative of HE in the direction r: 



HE(r + tr) = (HE" 

t=o 



The second derivative is a quadratic form in r. 
Theorem 4.20. 

f ( sec \ ■ 

HE = / r darea, 

JMr ^ COS a J 

HE"" = / 2 det darea + / r (cos a)"" sec darea . 

JMr JMr 

Proof. The first formula of the theorem follows by differentiating ()4.17p . 
The second formula is proved fully in the spirit of Theorems 14.151 and 14. 17^ 
by integrating the differential 2-form 

darea(Hess f A B). 

We have 

(4.22) / darea(Hess f A B) + j darea(V/ A B) = 0. 

J Mr JMr 
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Compute the first integrand: 

(4.23) darea(Hess f A B) = {h sec' -2h det B) darea . 

For tlie second integrand, observe that (j4.14p imphes 

dy B = J (-^ — Vr ) darea, 



.cos a 

due to the constancy of g and of the associated operators V and . Thus 
we have 

dareafV/ A B) = (V/, (^^\/r) ) darea 

\ VCOSQ / / 

(4.24) ^ ' 

'(V/,Vr)^ + (V/,Vr) ) darea. 

COS a \ cos Q / 

One easily computes (V/, Vr) = — /i(cosa)', hence 

(V/, Vr) = (V/, Vr)' = -/i(cosq)' - h{cosa)". 

By substituting this and (|4.20p in (|4.24p . we compute the scalar factor at 
darea: 

; / N- , / sec / . ■ \ / sec 
-n(cosa) — n,(cosa) I hlr — ncosa 



cosQ V / vcosa 

, , , , . sec / sec v\ , ... . / sec 

-n (cos a) h cos a — r(cos a) sec +r 

cos Q V cos a / / V cos a 

sec 



= —hsec' — r(cosa)" sec+r 

vcos a 
We finally obtain 

(4.25) darea(V/ A (i^i?) =( —/i sec' —r(cos a)" sec+r ( ) Idarea. 

\ V cos a / / 

By substituting (|i:23|) and (|i:25|) in (022]), we obtain 

/ I — 2/idet S — r(cos a) " sec +r ( ) ) darea = 

J Mr V Vcosa/ y 

Being combined with the first equation of the theorem, this implies the 
second equation. The theorem is proved. □ 

Proof of Theorem \3.3[ Let r = ro be the distance function of an embed- 
ding (/?o : — > IR'^ and let r be the variation of r that corresponds to an 
infinitesimal isometric deformation ^. By Theorem 14. 7|, we have sec' = 0. 
As the metric 5^0 is flat we have sec = 0. It follows that 

sec \ ■ sec' / 1 \ 

+ sec = 0. 



V cos a / cos a \ cos a 

Thus the first formula from Theorem 14.201 implies 

HE" = 0. 
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On the other hand, due to sec = the second formula says 

HE" = / 2hdetB darea. 

JMr 

Applying Corollary \A.6\ we obtain B = 0. 

It suffices to show that B = implies triviality of the deformation ^. Here 
B is understood as the variation of the shape operator of the surface 

Mr C (M+ X S^5v)• 

But, similar to the first part of the proof of Theorem 14.71 this is the same 
as the variation of the shape operator of the embedding 

With this interpretation of B, triviality of an infinitesimal deformation that 
preserves both the metric and the shape operator is well known. This can 
be seen as an infinitesimal version of the uniqueness part of the Bonnet 
theorem: first and second fundamental forms determine a surface uniquely. 
An alternative proof is to use the relation between B and the differential drj 
of the rotation field, see Lemmas 13.71 and 13.91 

Thus every isometric infinitesimal deformation is trivial, and the theorem 
is proved. □ 

5. Connections between Gauss and metric rigidity 

5.1. Shearing vs. bending. The proof of Theorem 12.41 in Subsection 12.31 
and Blaschke's proof of Theorem 13.31 in Subsection 13.41 are almost identical, 
although in the former t] is an isogauss infinitesimal deformation while in the 
latter r] is the rotation field of an isometric infinitesimal deformation. This 
similarity is explained by the following direct connection between Gauss and 
metric infinitesimal rigidity of surfaces in M^. 

Lemma 5.1. Let M C be a smooth surface. 

(a) //^: M — )■ is an isometric infinitesimal deformation of M , then 
its rotation vector field r] is an isogauss infinitesimal deformation 
ofM. 

(b) Conversely, if rj is an isogauss infinitesimal deformation of M and 
H^{M) = 0, then there exists an isometric infinitesimal deformation 
^ of M with rotation vector field r]. 

In both cases, ^ is the restriction of a Killing vector field if and only if r] is 
constant. 

Proof. Let ^ be an isometric infinitesimal deformation. By Lemma 13.61 its 
rotation vector field satisfies dr]{X) G T^M for ah X G T^M. By (fSTll . 
we also have tr{drj) = which means that rj, viewed as an infinitesimal 
deformation of M, preserves the Gauss curvature in the first order, see 
Lemma [2.51 Thus both conditions in Defintion 12.21 are satisfied and the first 
part of the lemma is proved. 
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In the opposite direction, assume that rj is an isogauss infinitesimal de- 
formation. Then we have ti{dr]) = 0. This implies that the vector-valued 
1-form r] X dp is closed. Since H^{M) = by assumption, there exists a 
vector field ^ along M such that dS, = rj x dp. Then, clearly, {dp, d^) = 0, so 
that ^ is an isometric infinitesimal deformation of M. By construction, rj is 
its rotation field, and the second part of the lemma is proved. 

The last statement of the lemma is contained in Lemma 13.71 □ 

Roughly speaking, it is the correspondence between Theorems 12.41 and 13.31 
described in the above Lemma that lead Blaschke to his proof. The matter 
is a bit complicated by the fact that Theorem 12.41 seems to not have been 
explicitely stated, neither before Blaschke nor by himself. Here is a detailed 
account of events. 

In the note [9] Blaschke observed that the rotation vector field of an 
isometric infinitesimal deformation satisfied a certain differential equation 
which, as Hilbert had shown in Section XIX], had only constant solu- 
tions. At the end of the paper, Blaschke promised to expand his argument 
and provide a geometric interpretation in a later article, but no such arti- 
cle appeared in the following years. Then Weyl [58] elaborated Blaschke's 
argument, including also the discrete case: infinitesimal rigidity of convex 
polyhedra. Blaschke rewrote this proof in a concise form in [10]; since then 
it became a classical argument and can be found i. a. in [53^ Chapter 12]. 

Hilbert dealt in [261 Section XIX] with Minkowski's theory of mixed vol- 
umes which involves deforming a convex surface by parallelly translating 
its tangent planes. A lemma in Hilbert 's work can be interpreted as the 
infinitesimal rigidity statement in Theorem 12. 4[ Minkowski worked mainly 
with convex polyhedra, and an analogous infinitesimal rigidity statement is 
Theorem 2.1 in [31] . See |3H Subsection 4.5] also for a discrete analog of 
Lemma 15. 11 

To explain the title of this Subsection, note that an isogauss infinitesimal 
deformation is shearing without bending while an isometric deformation is 
bending without shearing, and Lemma [5. 11 transforms bending into shearing. 

5.2. Polar duality between Gauss and metric rigidity. There is an- 
other connection between Gauss and metric infinitesimal rigidity. It relates 
metric rigidity of a surface with Gauss rigidity of its polar dual, as opposed 
to the previous subsection, where everything happens on a single surface. 

It will be convenient for us to change the setup and consider a surface 
M as parametrized by a map y?: S — > M^, where S is an abstract smooth 
surface. This extends the scope a little, as we are able to consider immersed 
surfaces instead of embedded ones. 

Everywhere in this subsection we assume the surface S to be orientable. 
Let z^: — ?• be a field of unit normals to M = i.p{S). Recall that 
h{x) = {v,(p{x)) is the support function of the immersion ip. We don't 
distinguish the map (p and the position vector p in this Subsection. 
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Definition 5.2. Assume that the support function of immersion ip: 5 — >■ M'^ 
nowhere vanishes. Then the map 

ip: S ^R^, 
h{x) 

is called the polar dual of ip. 

Geometrically, ij:{x) is the pole of the plane dip{TxS) with respect to the 
unit sphere in centered at the origin. 

The map ip is always smooth, but may fail to be an immersion. For 
example, if p maps an open subset U C S into a plane, then ijj maps all of 
U to the pole of this plane. 

Lemma 5.3. Let ^ be the polar dual of an immersion ip. If ip is itself an 
immersion, then the polar dual of is p. 

Proof. The condition on ip in Definition 15.21 is equivalent to 

(99,-0) = 1, {dip,ip) = 0. 

This determines the map ip uniquely up to a multiplication with —1 that 
corresponds to inverting the field of unit normals to if. 

By taking the differential of the first equation and subtracting the second 
one, we obtain {d^p, p) = 0. Together with the first equation (and under 
assumption rkd'i/' = 2) this forms the conditions on being the polar dual 
of ijj, under an appropriate choice of the field of unit normals to ■0- D 

Example 5.4. If 99: §^ — >■ is a smooth embedding with everywhere 
positive Gauss curvature and such that lies in the interior of the body P 
bounded by then the polar dual ■0 also enjoys all of these properties. 

If u in Definition 15.21 is the outward normal, then the body Q bounded by 
0(S^) can be described as 

Q = {-u; G I (u, w) < 1 for all v G P}. 

This is sometimes used as a definition of the polar dual of a convex body. 

Now we are ready to establish the announced equivalence. 

Lemma 5.5. Let 99: 5 ^ be an immersion such that its polar dual 
S* ^ M"^ is also an immersion. 

(a) //^: S* — > is an isometric infinitesimal deformation of M = (p{S), 
then its translation vector field r is an isogauss infinitesimal defor- 
mation of N = ip{S). 

(b) Conversely, if r is an isogauss infinitesimal deformation of N and 
H^{S) = 0, then there is an isometric infinitesimal deformation ^ of 
M such that r is its rotation vector field. 

In both cases, ^ is induced by a Killing field on if and only ifr is constant. 
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Proof. The part (jaj) is essentially proved in Lemma 13.101 There we have 
shown that {dT,ip) = and det((i^,(ir) = 0. The former is equivalent to 
dT{X) II TN , as p is orthogonal to TN , and together with the latter implies 
that the infinitesimal deformation r of iV preserves Gauss curvature in the 
first order, see Subsection 12. 2[ Thus both conditions of Definition 12.21 are 
fulfilled and r is an isogauss infinitesimal deformation of = ^(S"). 

In the opposite direction, assume that r: 5 ^> is given such that 
{dT,(p) = and det {dip, dr) = 0. It follows that the 1-form (ir x ^ is 
closed, and due to H^{S) = there is a vector field r^: S* — )• such that 
dr] = dr X ip. It follows that 

if X drj = (f X {dr x ip) = (93, ip)dT — {ip, dT)ip = dr. 

Thus if we put ^ = r]X(p + T, then ^ is an isometric infinitesimal deformation 
of M = ^p{S). 

The translation vector field of an isometric infinitesimal deformation is 
constant by definition. If r is constant, then the equation d-q = dr xip shown 
above implies that rj is also constant, and therefore ^ is trivial. The lemma 
is proved. □ 

5.3. Darboux wreath. By combining Lemmas 15.11 and 15.51 one can find 
another correspondence which is involutive up to sign. 

Lemma 5.6. Let S — > he an immersion such that its polar dual 
ip: S ^ is also an immersion. Assume that ^ is an isometric infini- 
tesimal deformation of M = ^p{S) with associated rotation and translation 
vector fields rj and r. Then the vector field 

(5.1) C = T X Ip — rj 

is an isometric infinitesimal deformation of N = ip{S) with r and rj as 
associated rotation and translation vector fields. 

Proof. By Lemma l5.ll r is an isogauss infinitesimal deformation of ^p. By 
Lemma 15.51 there exists an isometric infinitesimal deformation oi ip with 
rotation vector field r. To find the corresponding translation vector field, 
compute 

(5.2) ip X dr = ip X [(f X drj) = {tp, drj)ip — {ip, (p)dr] = —dr]. 

Thus by Definition in Subsection 13.21 —rj is the translation vector field of C- 
The lemma is proved. □ 

Assume that all maps from S to under consideration are immersions. 
Then r is an isometric infinitesimal deformation of rj with rotation vector 
field (p and translation vector field ^. In other words, the map 

(surface, deformation) 1— )• (rotation field, translation field) 

is an involution. On the other hand, one can view rj as an isometric infinitesi- 
mal deformation of r. Then, by equations (|5.ip and (|5.2p . the corresponding 
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rotation and translation vector fields are and — C. That is, the map 



has the orbit (9?,^) ^-^ (r, 7?) i-> (-C, -''A) By Darboux, [2T], gHl 

Section 3.4.1], the map (|5.3p has order six in general. The twelve surfaces 
in the orbit are called the Darboux wreath. 

6. Polar duality between volume and Hilbert-Einstein 

functional 

6.1. A conjecture about second derivatives. Lemma [531 establishes a 
correspondence between isometric infinitesimal deformations of a surface M 
and isogauss infinitesimal deformations of its polar dual N. In the case of 
convex closed surfaces, we identified isometric deformations with zeros of 
the second derivative HE' • of the Hilbert-Einstein functional, and isogauss 
deformations with zeros of the second derivative Vol' • of the volume. This 
suggests that there is a relation between the Hilbert-Einstein functional of 
a convex body P and the volume of the polar dual P* , at least on the level 
of second derivatives. 

Let M C be a convex closed surface with positive Gauss curvature 
and enclosing the coordinate origin, and let N be its polar dual. Denote 
by ro : K the distance function of M (see Definition 14.11 and by 

ho: S'^ — )■ M the support function of (see Subsection 12. 5p . Clearly, we 
have ho = l/rg. By equation (|4.17p . we have 



where g is the metric induced on by the radial projection (Pq: S'^ ^ M. 
Denote by Li the linearization at tq of the operator 



As the metric g remains constant during the warped product deformations 
considered in Section U the above formula implies that Li is self-adjoint. 
Similarly, we have 



(5.3) 



(surface, deformation) i-^^ (translation field, rotation field) 




sec 



r I— )• 



cos a 




see Subsection 12.51 Again, the linearization L2 of the operator 

h I— 7- det(/i id + Hess h) 



at h = Hq is self-adjoint. 



Lemma 6.1. kerLi = kerL2 
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Proof. As sec = at r = tq, we have kerLi = {r \ sec' = 0}. By Subsection 
14.11 there is an isometric infinitesimal deformation ^ of M such that 

r{x) = {^{x),x). 

By Lemma 15.51 the corresponding translation vector field r is an isogauss 
infinitesimal deformation of N. As det(/i id + Hess /i) is the reciprocal of 
the Gauss curvature of A^, it follows that the corresponding variation of h 
belongs to the kernel of L2- Since we have 

h{x) = (t(x),x) = (^(x),x) = r(x), 

where the second equation follows from ^(x) = i]{x) x x + t{x), it follows 
that ker Li C kerL2. By inverting the argument, we obtain kerL2 C ker Li, 
and the lemma is proved. □ 

The above lemma might seem less exciting, as the rigidity theorems imply 
that both kernels correspond to trivial deformations. But it says more when 
understood locally: note that Lemma 15.51 is of a local character, all 
other arguments in the proof of Lemma 16.11 

Conjecture 6.2. Operators Li and L2 are equal. 

This conjecture is motivated by a polyhedral analog, see [311 Lemma 4.1]. 

The rest of this section deals with spherical and hyperbolic-de Sitter geom- 
etry, where the relation between the Hilbert-Einstein functional and volume 
of the dual is a more straightforward one. 

6.2. Polar duality and fundamental forms in spherical geometry. 

Let S be an orientable smooth surface and S" — ?• S'^ be an immersion. 
We say that the surface A^ = <^(S^) is co- oriented, if for every x G S one of 
the half-spaces in which T^(^^^N divides T^(a;)S^ is dubbed positive, and this 
in a continuous way. A co-orientation can be introduced by chosing a unit 
normal field pointing in the positive direction. 

Definition 6.3. The polar dual of a co-oriented immersion ip is the map 
Tp: S" — )■ S'^ that sends x to the pole of the 2~sphere through ip{x) tangent to 
r^(2,)M. Of the two poles the one is chosen that lies on the positive side. 

Lemma 6.4. Let ip be the polar dual of an immersion (p: S ^ S"^. If ip 
is itself an immersion, then (p is the polar dual to ip, for an appropriate 
CO- orientation of N = ip{S). 

Proof. View §^ as the unit sphere in centered at the origin. For any 
v,w ^E>^ the scalar product {v,w) is then understood as the scalar product 
of corresponding vectors in M^. Then Definition 16.31 implies 

((^,V)=o, {d^,ip)=o. 

This determines the map ip uniquely up to antipodal involution which corre- 
sponds to changing the co-orientation of M = ifiS). Taking the differential 
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of the first equation and subtracting the second one yields dip) = 0. To- 
gether with the first equation and under assumption rkd^ = 2 this means 
that (f is the polar dual of ^, for a certain co-orientation of = ip{S). □ 

The polar dual can be interpreted as follows: ip{x) is obtained as the 
endpoint of the co-orienting unit normal 1^(2;) to 99, translated so that its 
starting point is at the origin of R'^. It fohows that 

which should again be understood as equality between free vectors in R^. 
Therefore the three fundamental forms of the immersion Lp can be written 
as follows: 

\^{X,Y) = {dip{X),d^{Y)), 
ll^{X,Y) = {d^{X),di^{Y)) = {d^{Y),di,{X)), 
lll^{X,Y) = {d^{X),dij{Y)). 

This immediately implies the following lemma. 

Lemma 6.5. Assume that the polar dual of an immersion ip is itself an 
immersion. Interpreting if as a unit normal field along ip, co-orient ■0 by (p. 
Then we have 

Iip = III^, 
II<^ = 11^, 
III<^ = . 

Finally, let us characterize immersions whose polar duals are also immer- 
sions. 

Lemma 6.6. The polar dual of an immersion ip has full rank at x € S if 
and only if <p{x) is not a parabolic point for <p. 

Proof. Choose an arbitrary volume form on S. This gives sense to determi- 
nants detl(^, detll(^, and detllli^. Then we have 

^^ = det(i?2) = (deti?)^ 
det ^^p 

where B is the shape operator of the surface M = (p{S). Thus the symmetric 
bilinear form = IIIi^ is non-degenerate if and only if det-B 7^ 0. The 
lemma is proved. □ 

In particular, if K(x) > 1 is the Gauss curvature of ip, then ip is an 
immersion with Gauss curvature K(x)~^. 
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6.3. Polar duality between Gauss and metric rigidity for surfaces 

in S^. Let ip: /S — )• S'^ be an immersion. An infinitesimal deformation of ip 
is a vector field along ip: 



7]-. S ^T8^, 

The geodesic flow of tjj along rj defines a family of immersions (for small t) 

Mx) = exp^(^)(t7?(3;)). 

Clearly, ipQ = ip. Denote 

d 



III 



dt 



III^, . 



The following definition is known to be the spherical analog of the Gauss 
infinitesimal rigidity defined in Subsection 12. 1[ 



Definition 6.7. A vector field rj along an immersion ip is called an isogauss 
infinitesimal deformation of N , iflll^ = 0. The surface N = ^{S) is called 
Gauss infinitesimally rigid if every its isogauss infinitesimal deformation is 
trivial, i. e. is a restriction of a Killing field on S^. 

In Gauss infinitesimal rigidity is straightforwardly related to the met- 
ric infinitesimal rigidity of the polar dual. 

Lemma 6.8. Let {ip, ip) he a polar pair of immersions of an orientable 
surface S in Then the surface N = Tp{S) is Gauss infinitesimally rigid 
if and only if the surface M = (p{S) is metrically infinitesimally rigid. 

Proof. Let rj be an infinitesimal deformation of q and 'pt = expg(t7/) be the 
corresponding geodesic fiow. Let ip'-f. be the polar dual of V't- By Lemma [ 
we have I^/ = III^^ , hence 



d_ 
IE 



t=o 



\[ — lllip- 



The left hand side depends only on the 1-jet of (p[, hence it does not change 
if we replace ip[ by ipt = exp^(t^), where ^(rr) = d/ dt\t=oipt{x). It follows 
that ^ is an isometric infinitesimal deformation of (p if and only if rj is an 
isogauss infinitesimal deformation of ip. 

If 7] is induced by a Killing vector field on S'^, then ipt = ° Q for the 
corresponding one-parameter group of isometrics $4. Then also ipt = ^t°P- 
It follows that ^ is induced by the same Killing vector field and is therefore 
a trivial deformation. The lemma is proved. □ 
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6.4. Herglotz's formula in S'^. The last subsection relates Gauss rigidity 
and metric rigidity of the polar dual in S'^ in a very natural way. Here we 
recall a formula of Herglotz that relates the Hilbert-Einstein functional to the 
volume of the polar dual. In the next subsection we discuss corresponding 
variational approaches to infinitesimal rigidity of surfaces in S^. 

Theorem 6.9 (Herglotz). Let P CE^ be a convex body with smooth bound- 
ary M with Gauss curvature bigger than 1. Let P* C S'^ be the body bounded 
by the surface M* polar dual to M . Then we have 

(6.1) Vol(P) + I H darea + Vol(P*) = vr^ 

Jai 

where H is the mean curvature, i. e. half the trace of the shape operator. 

Proof. Assume M to be co-oriented by the outward unit normal u, so that 
the bodies P and P* are disjoint (like northern and southern ice caps). We 
will compute the volume of the complement S'^ \{P U P*). 

The field of unit normals to M can be viewed as a map v. M ^ M*, 
and the differential dv can be identified with the shape operator B on M, 
cf. Subsection 16.21 Consider the map 

F: M X [0,f] ^ 

F{x, t) = cos t ■ X + sin t ■ ^{x) . 

Note that F(M x {t}) is the set of points at distance t from M and distance 
— t from M*. Thus the map F is a diffeomorphism onto the closure of 
§3 \ (p u P*) and we have 

Vol(S3 \{PUP*))= J F*(dvol§3). 

Mx[0,f] 

By denoting Ft = F(- , t), we can write 

F*(dvol§3) = Fj*(darea^j(^/)) = det {dFt) dtdareaM ■ 
As dFt = cos t ■ id + sin t ■ B, we have 

det{dFt) = cos^ t + 2smtcost ■ H + sin^ t ■ K. 

It follows that 

TT 

2 

Vol(S^ \ (P U P*)) = j darea j (cos^ t + 2 sin t cos i • iJ + sin^ t ■ K)dt 

M 

TT \ 

- + H + -K darea 
4 4 / 

M 



^Area(M) + y i7 darea +^ Area(M*). 



M 
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As Vol (§3) = 27r2, this leads to 

(0.2, Vol(P) + I A.ea(M) + / H da.e.+^ A,ea(M-) + Vol(p., = 2.^ 

M 

(an instance of Steiner's formula on §^). On the other hand, a simple com- 
putation: 



Area(M*) = j det((iF|) dareaM = j det-B dareaM 



/' 



M M 

{K - 1) dareaM = 47r - Area(M) 

M 

proves the formula 

(6.3) Area(M) + Area(M*) = Att. 

By multiplying it with 7r/4 and subtracting from (j6.2p we obtain (j6.1|) . The 
theorem is proved. □ 

Remark 6.10. There is an alternative to applying the Gauss-Bonnet for- 
mula in the second part of the proof. Consider the map 

G: Mx [0,f] ^ 

G(x, t) = cos t ■ X — sin t ■ i'{x). 

Then Gq maps M identically to itself, whereas 0,^/2 maps M to — M*. 
Attach a ball to each of the bases of the cylinder M x [0, ^] and extend the 
map G by mapping these balls to P and — P*, respectively. This results in a 
piecewise smooth map G : — ?• (the source space itself is equipped with 
only piecewise smooth structure). As G has degree 0, we have 

Vol(P) + j G*(dvol§3) + Vol(P*) = j G*(dvols3) = 0. 

A/x[0,f] §3 
By performing computations similar to the first part of the proof, we obtain 

(6.4) Vol(P) - ^ Area(M) + j H darea Area(M*) + Vol(P*) = 0. 

M 

By summing this with (j6.2p . we obtain (j6.ip . 

The argument that uses maps F and G is a slight modification of Her- 
glotz's proof in |25j . Herglotz works in S'^ for arbitrary d and obtains two 
formulas of which (j6.ip and (j6.3p are special cases for d = 3. For an odd d 
both formulas are self-dual (in particular, J^^ iif darea is also the total mean 
curvature of M*), while for an even d they are dual to each other, so that 
the terms Vol(P) and Vol(P*) occur in different formulas. 



44 RIGIDITY OF SURFACES AND THE HILBERT-EINSTEIN FUNCTIONAL II 



Remark 6.11. Points of P* are poles of great spheres disjoint with the 
interior of P. This provides an integral-geometric interpretation of the for- 
mula (j6.ip : a random great sphere intersects a convex body P with the 
probability 



^ I Vol(P) + j H darea j . 
\ dP / 



IT 

6.5. Approaches to proving infinitesimal rigidity of convex surfaces 

in A smooth convex closed surface M C with Gauss curvature big- 
ger than 1 is infinitesimally rigid, see Remark 16.151 below. Most likely, a 
direct proof can be found that uses the approximating section in the bun- 
dle of germs of Killing fields, similarly to the rotation and translation fields 
approach in Subsections I3.2H3.51 The proof from Subsection 13.51 should be 
equivalent to studying derivatives of the functional 



S{P) := 2 Vol(P) + ^ j seal dvol-F2 j H dar 



ea, 



p dP 

where P is the body bounded by M, and the metric in the interior of P 
varies in the class of warped products, cf. Section HI The functional S can 
be seen as a gravity action with non-zero cosmological constant, cf. |34] . 

Under the above assumptions, the surface M is also Gauss infinitesimally 
rigid. As indicated by Subsections 16.31 and 16.41 a proof of (metric) infinitesi- 
mal rigidity of M using the Hilbert-Einstein functional should translate in a 
straightforward way as a proof of Gauss infinitesimal rigidity of M* = dP* 
using the volume of P* . One should probably study the functional 



S*{P*) := 2 Vol(P*) + seal dvol. 



P* 

that is the gravity action without the boundary term, where the metric in 
the interior of P* varies in the class of warped products while preserving the 
third fundamental form of the boundary. Indeed, warping the metric around 
the north pole of §^ that is contained in P is equivalent to warping around 
the south pole contained in P*, and we conjecture that S{P) + S*{P*) = 2tt'^ 
holds for every warped product metric on S^, cf. [3H Lemma 4.9]. 

6.6. Polar duality in hyperbolic-de Sitter geometry. Less classical as 
the polar duality in the sphere is the polar duality between the hyperbolic 
and de Sitter spaces, see dH Section 1] and references therein. 
Consider the hyperboloid model of the hyperbolic space 

= {x e M'^ I ||2;||3,i = -l,xo > 0}, 

where |Ix||3^i = —Xq + + + Xg. The polar dual to an immersion 
ip: S ^ is an immersion in the de Sitter space 

dS^ = {x € I ||x||3,i = 1}. 
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Indeed, if we define the polar dual ^p similarly to Subsection 16.21 through 

then ijj{x) lies on a space- like line, which intersects dS^ and not H^. 

Relations between the fundamental forms and the Gauss and metric in- 
finitesimal rigidity for polar pairs of surfaces carry over to the hyperbolic-de 
Sitter case. In order to transfer Theorem 16.91 one has to give meaning to 
the term Vol(P*). First of all, we put 

P* = cone(M*) n (dS^ U M^), 

where cone(M*) C M"^ is the cone over M*, and IHlJ^ is the antipodal copy 
of H^. Thus P* is the union of H'i and of an infinite end of dS'^. However, 
there is a consistent way to define a finite quantity Vol(P*). 

Theorem 6.12. Let P CM^ be a convex body with smooth boundary M with 
everywhere positive definite shape operator. Let P* C dS^ be the convex body 
bounded by the surface M* polar dual to M . Then we have 

(6.5) Vol(P)-/" i7 darea + Vol(P*) = 0, 

where H is the mean curvature, i. e. half of the trace of the shape operator. 

Herglotz [25] proves this theorem in the same way as in the spherical 
case, multiplying the arguments of sin and cos with i. In footnote 10, he 
remarks that the same result can be achieved by studying the asymptotics 
of the volume of parallel bodies in 'M?. In [19], contour integrals in the 
complex plane are used in order to assign a finite volume to certain subsets 
of U dS"^. This might be related to the argument of Herglotz. 

Remark 6.13. An integral geometric interpretation of ()6.5p says that the 
(motion-invariant and appropriately normalized) measure of the set of planes 
that intersect a convex body P C equals 

j H darea-Vol(P). 

dP 

In particular, this quantity is always positive. 

Remark 6.14. In [34], the asymptotics of the volume of parallel bodies of 
the convex core is used to define the renormalized volume of a non-compact 
hyperbolic manifold. 

Remark 6.15. Infinitesimal rigidity of smooth surfaces is invariant under 
projective transformations, as was shown by Darboux [21]. Closely related 
to this are the so-called Pogorelov maps, [441 Chapter 5]. They associate 
to an isometric infinitesimal deformation of a surface M C M"^ an isometric 
infinitesimal deformation of the surface Me C 'M? obtained by taking an 
open ball B D M and interpreting it as the Klein model of M^. As a result, 
-Mh is infinitesimally rigid if and only if M is. The same holds with Mm 
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replaced by M§ C defined as the inverse gnomonic projection of M. 
Volkov in [55] gives a unified treatment of these two results. 

A surface M C is convex if and only if Me (respectively, M§) is 
convex. Thus infinitesimal rigidity of closed strictly convex surfaces in the 
hyperbolic (respectively, spherical) space follows from the rigidity in the 
Euclidean space. 



7.1. A brief overview. The following problem was posed by Weyl in |57j . 

Weyl problem. Let g be a Riemannian metric on with everywhere pos- 
itive Gauss curvature. Show that there exists a smooth convex embedding 
— > such that ip*{can]^3) = g. Show that this embedding is unique 
up to an isometry o/M"^. 

Weyl outlined a proof for the analytic case; it was accomplished later 
by H. Lewy [38] • Nirenberg [41] extended Weyl's method to certain finite 
differentiability classes. 

A. D. Alexandrov [H S] stated and proved a polyhedral analog of the 
Weyl problem. In this case 5 is a Euclidean metric with cone points of an- 
gles less than 2tt, and (S^,g') must be embedded as a convex polyhedron. 
By approximating a Riemannian metric with polyhedral ones, Alexandrov 
showed that {S'^,g) from the Weyl problem can be embedded isometrically 
into M.^ as a convex surface, but he hasn't shown that the embedding is 
smooth. Pogorelov filled this gap in [l5], again for some finite differentiabil- 
ity classes, and strengthened his results in later works. 

Minkowski problem. Let K : ^ (0, +00) be a smooth function such 
that 



where = {x G | = 1}. Show that there exists a smooth convex 
embedding tp: — )■ such that K{x) is the Gauss curvature of the surface 
N = ip{S'^) at the point tp{x), and x is the outward unit normal to N at 
ip{x). Show that the embedding ip is unique up to a parallel translation. 

The problem that was stated and proved by Minkowski in [lO] is differ- 
ent: he assumed only continuity of K and wanted to prove the existence 
of a convex surface whose curvature measure has density K, the curvature 
measure being defined as the measure of the Gauss image pulled back to 
A^. Minkowski first proved an analog for convex polyhedra and then used 
polyhedral approximation. Later, the argument was extended to arbitrary 
measures, not necessarily having a positive continuous density, [2], [50l Sec- 
tion 7.1]. Pogorelov [46] proved that if K is smooth then the corresponding 
surface is also smooth and thus has Gauss curvature K. This settled the 
Minkowski problem in the formulation given above. 



7. Relation to Minkowski and Weyl problems 
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In a different line of researcli, H. Lewy [37] solved the Minkowski problem 
for analytic metrics by adapting the method suggested by Weyl for solution 
of his problem, and Nirenberg |41j extended this to the smooth case. 

7.2. Uniqueness in the Minkowski and Weyl problems. The unique- 
ness part in both Minkowski and Weyl problems is essentially easier than the 
existence part. For the former, it was proved by Minkowski himself using 
his mixed volumes theory. Minkowski's ideas were developed by Alexan- 
drov, who gave another uniqueness proof using mixed determinants, [T]. 
The uniqueness in the Weyl problem was proved by Herglotz in [24] ; this 
proof is reproduced in |27j . 

A proof similar to that of Alexandrov was found by Chern [T7] and, inde- 
pendently, by Hsiung [29] . The main difference is that they argued in terms 
of the position vector of an embedding while Alexandrov argued in terms 
of the support function. The proof by Chern and Hsiung is reproduced in 
[53\ Chapter 12]. One should also mention a paper [18] of Chern, where he 
generalizes his arguments and establishes a connection with Alexandrov's 
work. 

Quite curiously, the ideas came full circle. Hsiung' work was motivated 
by that of Herglotz. Herglotz refers to Blaschke's proof of the infinitesimal 
rigidity of convex surfaces. And Blaschke's proof was inspired by Hilbert's 
work [26] related to the Minkowski problem, see discussion at the end of our 
Subsection 15.11 

Our proof of the infinitesimal Gauss rigidity in Subsection 12.31 is related 
to Chern-Hsiung's proof of the Minkowski uniqueness in the same way as 
Blaschke's proof of the infinitesimal metric rigidity is related to Herglotz's 
proof of the Weyl uniqueness. However, we came to our proof by "smooth- 
ing" the polyhedral analog, [3T] . 

7.3. Existence in the Minkowski and Weyl problems. Weyl, Lewy, 
and Nirenberg solved the Minkowski and Weyl problems using the continuity 
method. For example, in the case of Weyl's problem, they consider a family 
of metrics {gt\t ^ [0, 1]} on such that gi = g and go is the standard metric 
of curvature 1. For i = there is an isometric embedding {S^,gt) — )• M"^, 
and one wants to show that the set of all t for which this is the case is an 
open and closed subset of [0, 1]. The openness follows from the ellipticity of 
a certain operator (which is, in fact, related to infinitesimal rigidity), and 
closedness follows from a priori estimates. 

Alexandrov's solution of the polyhedral Weyl problem also uses a variant 
of the continuity method. At the same time, Minkowski proved the polyhe- 
dral Minkowski theorem in a different, quite elegant way. On the space of 
convex polyhedra with given directions of outward normals, he maximized 
the volume function under a certain linear constraint and showed that the 
maximum point is the desired polyhedron up to a scaling. Alexandrov [2] 
extended this method to arbitrary curvature measures. 
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A variational approach to the Weyl problem was suggested by Blaschke 
and Herglotz in [TTj. Their idea was to consider it not as an embedding 
problem (§^,g) (M^, can), but as an extension problem: given a ball B'^, 
a Riemannian metric g onS"^ = dM^ must be extended to a flat metric on B'^. 
On the space of all extensions g they consider the Hilbert-Einstein functional 
and show that its critical points are exactly the flat metrics. Thus the Weyl 
problem reduces to showing that the Hilbert-Einstein functional has exactly 
one critical point. This idea was never implemented. 

7.4. An approach to the Minkowski and Weyl problems. SectionSlof 
the present paper suggests a modification of Blaschke-Herglotz's approach: 
instead of considering all Riemannian metrics g on B^ that extend a given 
metric g onS^, one considers only warped products gr of the form ()4.3p . The 
metric gr is determined by a function r : — ?■ M and determines a function 
sec: — )• M such that gr is flat if and only if sec = 0. We have shown in 
Section m that the kernel of the map r i— t- sec' (the linearization of r i— t- sec) 
at r such that gr is flat has dimension three. 

Conjecture 7.1. The map r i— > sec" is onto provided that 

(7.1) < sec < K, 

where : §^ — )• M is the Gauss curvature of the metric g. 

(Note that sec < K implies det > 0, where B is the shape operator on 
embedded in the warped product.) 

If this conjecture is true, then under assumption (j7.ip every small pertur- 
bation of sec can be achieved by an appropriate modification of r. 

In order to achieve sec = 0, put r^ = R for a sufficiently large i? € M. 
Then the corresponding seco satisfies (j7.ip . Construct a family of functions 
{rt\t ^ [0, 1)} by requiring 

(7.2) sect = (1 — i) seco . 

One needs some a priori estimates in order to show that the set of t € [0, 1) 
for which rt exists is closed and that the limit at t — )• 1 is a smooth function. 
This would prove the existence in the Weyl problem. 

In the polyhedral case, the above approach was realized in [12]. A different 
modification of the Blaschke-Herglotz approach in the polyhedral case was 
found and realized by Alexandrov's student Volkov in |54| . 

A similar approach to the existence part of the Minkowski problem con- 
sists in choosing an arbitrary function /iq : — )■ M and constructing a family 
of functions {ht\t £ [0, 1]} such that K^^ - R-^ = (1 - t){K^^ - K'^), 
where Kt is the Gauss curvature of the surface with support function hf. 

In [20], the curvature flow h = —log{Kt/K) is considered. It is shown 
that, depending on the initial data, the solution either shrinks to a point or 
expands to infinity or converges to a smooth surface with Gauss curvature K. 
The advantage of this approach is the expliciteness of the evolution equation. 
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while our approach would give convergence to the solution of the Minkowski 
problem independently of the initial data. 

8. Directions for future research 

Problem 1. What is the analog of Darboux wreath (Subsection 15. 3p for 
infinitesimal deformations of surfaces in the sphere and in the hyperbolic-de 
Sitter space? 

Problem 2. Find new proofs of the existence part in the Minkowski and 
Weyl problems based on the approach outlined in Subsection 17.41 

Koiso in [33] proved infinitesimal rigidity of Einstein manifolds under 
certain restrictions on the curvature by studying the second variation of the 
Hilbert-Einstein functional. Our approach to the infinitesimal rigidity of 
convex surfaces in Section U] has some similar features. 

Problem 3. Prove infinitesimal rigidity of an Einstein manifold M with 
convex boundary by unifying Koiso's and our arguments. 

Koiso used certain Weitzenbock-type formulas for HE". In the case with 
boundary an integral over dM should appear. Hopefully it can be identified 
as Jqj^j f det B dvol for some positive function /. 

Koiso's method works in particular for compact closed hyperbolic mani- 
fods (whose infinitesimal rigidity was previously proved by Calabi [1^ and 
Weil [56]). Thus a special case of Problem [3] is to give a new proof of 
Schlenker's theorem [39] on the infinitesimal rigidity of compact hyperbolic 
manifolds with convex boundary. Schlenker proves also Gauss infinitesimal 
rigidity of such manifolds. It would also be interesting to find a variational 
proof of this theorem, cf. Subsection 16. 5i 

Similar problems for manifolds with convex polyhedral boundary are not 
solved, although infinitesimal rigidity of convex polyhedra can be proved in 
a similar spirit, cf. [31]. A challenge would be to find a common generaliza- 
tion from polyhedral and smooth to arbitrary convex surfaces (respectively, 
manifolds with convex boundary). 

Problem 4. By extending the approach proposed in Problem [S] prove the 
infinitesimal rigidity of compact hyperbolic manifolds with convex boundary 
which is neither smooth nor polyhedral. In particular, prove the infinitesimal 
rigidity of the convex core of a hyperbolic manifold. 

Infinitesimal rigidity of convex cores would probably allow to prove the 
uniqueness part in Thurston's pleating lamination conjecture. See |14j . 
where the existence part is proved. 

Note that the Schlafli formula for convex cores proved by Bonahon [13] 
should be equivalent to a formula for the first variation of the Hilbert- 
Einstein functional under arbitrary variations of the metric in the interior 
of the convex core. More generally, first variations of Lipschitz-Killing cur- 
vatures in the case of non-smooth boundary are computed by Bernig in [8] . 
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Problem 5. In the same spirit, reprove Pogorelov's theorem |441 Chapter 
IV] on the infinitesimal rigidity of arbitrary convex surfaces without flat 
pieces by studying variations of the Hilbert-Einstein functional. 

Appendix A. Mixed determinants of linear operators and of 

VECTOR- VALUED DIFFERENTIAL FORMS 

A.l. Mixed determinants of linear operators. Let V be an vector 
space with diml/ = n. Then the determinant 

det : End{V) R 

is a homogeneous polynomial of degree n. Therefore there is a unique po- 
larization of det, that is, a symmetric polylinear form on End(y) denoted 
also by det and such that 

det{A, ...,A)=detA 
for all A G End(y). See e. g. [Ml Appendix A]. 

Definition A.l. The number det(^i, . . . ,An) is called the mixed determi- 
nant of operators Ai, . . . , An . 

For example, by polarizing the determinant of 2 x 2-matrices we obtain 
det ( '''A , (I'' I'A) = }-{anb22 + a,2bn-a,,b,,-aM. 

V V"21 a22 J V"21 022/ J 2^ 

Corollary A. 2. 

det (id, 5) = ^tTB 

Lemma A. 3. The symmetric bilinear form det(- , •) on the space o/ 2 x 2- 
matrices has signature {+, +, — , — ). The restriction of det {■ , •) to the space 
of symmetric 2 x 2-matrices has signature (+,—,—). 

Proof. The matrices 

form an orthogonal basis for det(- , •), the first three of them spanning the 
space of symmetric matrices. The first and the fourth matrix have a positive 
determinant, the second and the third a negative. □ 

Corollary A. 4. Let A, B be symmetric 2 x 2-matrices such that 

(A.l) det B>0, det{A, B) = 0. 

Then det^ < 0. Besides, the equality det^ = occurs only if A = 0. 

Proof. Assumptions (lA.ip mean that i? is a positive vector for the symmetric 
bilinear form det(- , •) and that A is orthogonal to B. By the second part of 
Lemma |A.3( det(- , •) is negative definite on B^. Hence det74 < 0, and the 
determinant vanishes only if the matrix A vanishes. □ 
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Lemma A. 5. Let Bt € End(F) he a differentiable family of linear operators. 
Denote B = Bq and B = ^|^_q Bt. Then 

{det BY = 2det{B,B), 

where {det B)' = ^ (det St). 

Proof. Follows easily from bilinearity and symmetry of det(- , •). □ 

Corollary A. 6. Let Bt be a differentiable family of symmetric 2x2- 
matrices, Bq = B. Assume that 

detB>0, (detB)"=0. 

Then detB <0. 

Besides, the equality detB = occurs only if B = 0. 

Proof. Use Lemma I A. 51 and Corollary IA.4I for A = B. □ 

A. 2. Mixed determinants and tensor algebra. Recall the definitions of 
the alternating and symmetrizing operators on the k-th tensor power V 
of a vector space V: 

Alt(vi (g) • • • (g) Ufc) := ^ sgno- • v^^i^ ® • • • (g) w^(fc), 

Sym(?;i ^ ■ ■ ■ (g) Vk) := ^ ^ i^a(i) <8) • • • u^(fc). 

The image of Alt is denoted by /\^ y C 0^ V and consists of linear 
combinations of multivectors 

vi A-- - Avk := /c! Alt(f 1 (g) • • • (g f fc) = ^ sgn a ■ v^(^i) (g • • • (g ^^-{fc) • 

Let Ai, . . . , Ak G End(y) = F (g y* be a collection of linear operators. 
Their tensor product acts naturally as a linear operator on the fe-th tensor 
power of V: 

k 

(A.2) (9) e (g)'=(y ® V*) ^ ((g)V) (g (^''{V*)) ^ End ((g)V^ 



i=l 

because we have ^''{V*) ^ (^''vY. Explicitly, 



(vi (g • • • g) Vk) := Ai{vi) g) • • • g) Akivk). 



.1=1 



If Ai = A for all i, then the operator A commutes with Alt: 
((g)'^^) (Alt(ui®---0Wfc)) = ^ sgnCT- A(t.„(i))(g---® 



Alt (((g)''^) (Ul ® • • • (g Wfc)) . 
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Consequently, 

((g)^A) {viA---Avk)= A{vi) A • • • A A{vk), 

so that the operator A maps /\^ V to itself. In particular, if dimy = n, 
then dim/\"y = 1 and A\^ny is a multiplication by a scalar. It is 
well-known that this scalar is the determinant of A: 

(A.3) (g)"^|^„^ = det^-id. 

Now let's come back to the general case (|A.2p . The operator (^^^^ in 
general does not commute with the alternating operator. Instead, consider 
the symmetrized tensor product 

Sym(g)A, = l j;(g)^.(,). 



kl 

i=i o-eSfe i=i 



that is the image of (^^=1 Ai under the symmetrizing operator 

Sym: ^''{V V*) <S)''{V V*). 

A simple computation shows that Sym(^^^^Aj commutes with the alter- 
nating operator on V and acts on /\'' V as 



k 



I Sym I^aA {viA--- Avk) = ^Yl [^^''^ ' MiMd))] 

\ i=l J ■ a,T€Sk V i=l / 

For k = n = dimT^ we have the following lemma that can serve as an 
alternative definition of the mixed determinant. 

Lemma A. 7. If d\mV = n and Ai,. . . ,An € End(y), then 



(A.4) Sym(^^, = det(Ai, . . . , A) • id . 

V i=i / A"y 

Proof. lfAi = A for all i, then we have Sym (g)" A = (g)" A, and (|A.4p turns 
into (IA.3p . Both sides in (IA.4P are symmetric and polylinear in (Ai): the 
left hand side by construction, the right hand side by definition. Therefore 
the equality holds for all {Ai). □ 

Remark A. 8. Instead of applying symmetrization to ^^^^Ai, one can 
compose alternation with (g)^!^^ A^. This gives the same operator when re- 
stricted to /\'' V: 



(A.5) 







^Alto(g)Ai^ 


= [sym^^i'j 
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A. 3. Vector-valued forms. Let V and W be vector spaces. 

Definition A. 9. A V -valued k-form on W is an element of V /\^ W* . 
For two V -valued forms w G F (g /\ W* and rj € V /\ W* , their wedge 
product 

uArie {V ^V)(^ f\''^^W* 
is defined as the image of oj r] under the map 

id (g) A : (y ® y) (g) (^/\''W* (g A'W^*) ^{V(^V)(g> f\^^^W* 

that takes the wedge product of /\W* -components. 

The wedge product of several ^-valued forms is defined in a similar way. 
Let n = dimF, and let dvol G A"(^*) be a distinguished element, a 
volume form. 

Definition A.IO. Let uj G ((g)" F) ® /\^W* he a {<S)''^ V) -valued k-form 
on W. Define a k-form 

dvoKw) G fX^'W* 
as the image of u under the linear map 

dvol (g id: ((g)"y) (a'^w^*) ^ A'^w*. 

In particular, for G ^ (g> A^' i = 1, . . . , n we have a fc-form 

dvol(a;i A • • • A a;„) G f\''W, 

where k = Y17=i^i- ^ special case V = W,ki = 1 this construction is 
related to the mixed determinant of linear operators. Indeed, each 

is a linear operator on V, and we have the following lemma. 

Lemma A. 11. Let G ^ (g) V* , i = 1, . . . ,n, be V-valued 1-forms on an 
n-dimensional vector space V . Then 

(A. 6) dvol(a;i A • • • A w^) = n! det(a;i, . . . • dvol, 

where on the right hand side stands the mixed determinant of linear operators 
Ui G Endy. 

Proof. The proof goes by inverting the roles of V and V* in Lemma lA.71 and 
Remark IA.8I 

Since V 0V* = End(V^*), we can view as a linear operator on V*. It 
is easy to see that 

n 

A • • • A w„ = n! Alt o (g) G End ((g)"^*) . 

i=l 

Due to ()A.5|) and (IA.4P , the restriction of wi A • • • A to A" ^* is multi- 
plication with n! det(a;i, . . . ,a;„). On the other hand, dvo^wi A • • • A LOn) is 
by definition the value of wi A • • • A on dvol. Equation (IA.6[) follows. □ 
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More generally, if G F (g) W* = Hom(VF, V) with dim = dimV = n, 
then we have 

dvol\/(u;i A • • • A Un) = n\ det(a;i, . . . , ujn) ■ dvolw, 

where dvoly and dvolvy are some volume forms on V , respectively W ^ and 
the mixed determinant is the polarization of det : Hom(T4^, F) — t- M which 
is defined with respect to dvoly and dvol^y. In the special case Wj = a; we 
obtain 

(A. 7) dvoly (A"'^) — det bj ■ dvolvK ■ 

One sees immediately that 

dvoly {fS^oj) = n!a;*(dvoly). 
Therefore equation (jA.Tp is just another form of a well-known identity 

cj* (dvoly) = det a; • dvol^y . 

Lemma A. 12. Let LOi G V* be a V -valued form on V , i = 1, . . . ,n. 

Then we have 

dvol(a;i A A • • • A = (-l)'^'i^2+i dvol(a;2 A A • • • A a;„), 
and similarly for any other transposition of two factors. 

Proof. As with usual forms with values in a field, transposing two factors 
in A • • • A a;„ exchanges two blocks of factors in /\^ V* which causes the 
factor (— to appear. In our case there is also a transposition of two 
factors in V which changes the sign of the evaluation of dvol. □ 

A. 4. Vector bundle-valued differential forms. Let M be a smooth 
manifold, and let ii^ — ?■ M be a smooth finite-dimensional vector bundle 
over M. 

Definition A. 13. An E'-valued differential /c-form on M is a section of the 
vector bundle E ® f^{T*M). The set of all E-valued differential k~forms 
is denoted by n''{M,E). 

The wedge product of two vector bundle- valued differential forms 

A: n^{M,E) (g)n^ (M, F) (M, E(E)F), 

is defined as in Definition IA.9[ If dim E = n, and E is equipped with a 
volume form dvol G r(/\"£'), then we define 

dvol: n'' {M,^''E) VL^{M) 

as in Definition lA.lOl 

A connection V on E induces exterior differentiation of E'-valued differ- 
ential forms: 

d^: Q''{M,E) Q''+\M,E), 

see e. g. [321 Chapter 3.1]. A connection V induces also connections on 
all tensor bundles associated with E, and thus exterior differentiation of 
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corresponding tensor bundle-valued differential forms. All these definitions 
go by postulating certain Leibniz rules and imply the following two lemmas. 

Lemma A. 14. Let uji € ^'''■{M,E), i = l,...,p be E-valued differential 
forms on M . Then we have 

p 

d^{uji A--- Aujp) = ^(-l)''i+'''+'''-i(a;i A • • • Ad^Ui A • • • AWp), 

i=l 

where d'^ are the exterior derivatives on Q{M,E) and i}{M,^^ E). 
Lemma A. 15. Assume that the volume form dvol is parallel: 

V(dvol) = 0. 

Then we have 

d{dvol{u)) = dvol {d^u) 
for every uj G n^{M, (gl'^E). 

In particular, the assumption of Lemma IA.15I holds if the vector bundle 
E is equipped with a scalar product, dvol is the associated volume form, 
and V is a metric connection. A natural situation when this occurs is a 
smooth submanifold M of a Riemannian manifold N with E = TN\m, the 
restriction of the tangent bundle of N to M. More specifically, N may be 
M itself, with V the Levy-Civita connection on E = TM. 

A. 5. Vector-valued diflferential forms. Assume that E is trivial and a 
trivialization 

(A.8) E^V X M, 

is fixed. This induces an isomorphism 

n^{M,E) ^ C^{M, V) (g) n'^M 

which allows us to call differential forms with values in a trivialized bundle 
vector-valued differential forms. A choice of a basis in V establishes an 
isomorphism 

C7~(M, V) ® n'^M ^ (fi'^M)", 

where n = dim V. 

Let V be the connection associated with the trivialization (jA.Sp . The 
associated exterior derivative d^ is nothing else than the componentwise 
differentiation 

with respect to an arbitrary choice of a basis in V. 

A trivialization (lA.Sp induces also a trivialization of the dual bundle E*. 
Choose an element dvol S /\" V* and consider the constant section dvol of 
the bundle /\" E*. Then we have V(dvol) = 0, which implies that Lemma 
IA.15I holds for the canonical connection on a trivialized bundle. 

The next lemma states that a trivial connection if flat, which is false for 
general connections. 
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Lemma A. 16. Let be the exterior derivative associated with the canon- 
ical flat connection on a trivialized vector bundle (jA.Sp . Then we have 

d^{d^uj) = 0, 

for every V -valued differential form u. 

Proof. Immediate from the interpretation of as the componentwise exte- 
rior differentiation. □ 

A natural situation when we have to do with a triviahzed vector bundle 
is an embedded manifold M C M" with the vector bundle E = TW^\m- 

Appendix B. Curvature of p^-warped product metrics 
Let 5 be a Riemannian metric on Consider the Riemannian metric 
(B.l) g = dp^ + p^g 

on the manifold M+ x S^. In particular, the metric gr in (I4.3P is of this form. 

We compute here the Riemann tensor and the sectional curvatures of the 
metric g. In more general settings, this is done in [421 Section 7.42] and |35t 
Section 6, Exercise 11]. 

Let R be the curvature tensor of the Riemannian manifold (M+ x S^,^). 
Let dp be the unit vector field arising from the product structure on x S^. 
By C r(p ,j,)(M+ x we denote the plane orthogonal to dp. As dp is 
also the gradient of the function p, the plane is tangent to the level 
sets of p. Denote by sec^p^^.) (L) the sectional curvature of g in the plane 
Lcr(p,,)(M+ xS2). 

Lemma B.l. The curvature of the Riemannian metric (jB.ip has the fol- 
lowing properties. 

(B.2) R{X, Y)Z = sec(a^) • dvol(ap, A, Y) ■ {Z ^ dp) 

for all vectors X,Y,Z e r(p^^)(R+ x S^); 

(B.3) sec(p^^)(L) = cos^ if ■ sec(p_^)((9^), 

where tp is the angle between the planes L and d^; 

I seen T)(d^) 
(B.4) sec(p,.)(ap^)= 

Proof. The curvature tensor can be computed following |431 Chapter 2.4]. 
For this, consider the function p on M_|_ x — )• M, given by the projection 
to the first factor. This is a distance function, i.e. its gradient has norm 1. 
The Hessian 

Hessp : X i-^ VxVp 

can easily be computed: 

Hessp = —TTp, 
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where vTp is the orthogonal projection to dp. Then the radial curvature 
equation yields 

R{-,dp)dp = Q, 

and the mixed curvature equation (Codazzi-Mainardi equation) implies 

g{R{X,Y)Z,dp) = Q, 
for any vectors X,Y,Z G 5^ . It follows that 

(B.5) g{R{X, Y)Z, W) = g{R{X' , Y')Z', W'), 

for arbitrary vectors X,Y, Z,W , where X' = TTp{X) and so on. As X' , Y', 
Z', and W all lie in the plane d^, we have 

(B.6) g{R{X',Y')Z', W') = sec{d^) ■ darea(X', Y') • darea(Vl^', Z'), 
where darea denotes the area form in induced from g. Clearly, 

darea(X',y') = dvol{dp, X,Y), 
darea(VF', Z') = dvol(5p, W, Z) = g{Z x dp, W). 
By substituting this in (|B.6h and using (fR5|l . we obtain ([Bj]). 

□ 
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